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Preliminary Announcement—Symposium on Information Theory 


The Professional Group on Information Theory of the Institute of Radio 
Engineers, in cooperation with the Research Laboratory of Electronics of the 
Massachusetts Institute of Technology, is planning to hold a Symposium on 
Information Theory at the Massachusetts Institute of Technology, Cambridge, 
Mass., on September 10-12, 1956. The Symposium will be co-sponsored by the 
Office of Naval Research, the Air Research and Development Command, the 
Signal Corps Engineering Laboratories, and the U.R.S.I. 

A similar symposium was conducted by the same organizations in September, 
1954. It is the intention of the Organizing Committee to follow the pattern of the 
1954 meeting, and to make this symposium an occasion at which authorities in this 
field may present and discuss the significant advances which have been made 
during the year. In order to provide an opportunity for informed and creative dis- 
cussion of the papers, the committee is planning to distribute the Symposium 
Transactions two weeks prior to the meeting, as was done for the 1954 Symposium. 

The Symposium Transactions will be distributed as a regular PGIT publica- 
tion and copies will be available to the members of the co-sponsoring organizations 
at the same price as to all IRE members. 

Submission of papers is hereby invited. In order to carry out successfully the 
advance publication plan, abstracts of the papers should be submitted at the 
earliest possible date. The form of the Symposium Transactions will be the same 
as that used for the IRE CONVENTION RECORD and authors will be expected 
to prepare a final copy of their papers on special format suitable for photo-offset 
reproduction. The material and instructions for the preparation of this final manu- 
script will be mailed to the author at the time of acceptance of the abstract or at 
any time upon request to the Chairman of the Organizing Committee: P. Elias, 
Research Laboratory of Electronics, Massachusetts Institute of Technology, 
Cambridge 39, Mass. 

The deadline schedule for abstracts and full-length papers is as follows: 


May 1—Deadline for receipt of 750-1,000 word abstract of proposed 
paper. 

May 15—Authors notified of acceptance or rejection of papers submitted 
in abstract form. 

June 15—Deadline for reception of full-length papers not previously 
submitted in abstract form. These must be submitted on 
special paper and in the approved format. 

July 1—Deadline for reception of final master copy of papers which have 
been previously accepted in abstract form. 

Sept. 1—Mailing of Symposium Transactions. 


The above deadlines are rigid because of the publication schedule. Early sub- 
mission of abstracts will be greatly appreciated by the Organizing Committee. 


Annual ACM Meeting 


The Annual Meeting of the Association for Computing Machinery will be 
held at the University of California Westwood Campus, Los Angeles, on August 
27-29, 1956. For information, write to G. W. King, Box 3251, Olympic Station, 
Beverly Hills, Calif. Submit papers (abstract and four page manuscript in triplicate) 
by May 15 to J. P. Nash, University of Illinois, Urbana, Ill. (See January issue, 
Journal of Association for Computing Machinery for further details.) 
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CLAUDE E. SHANNON 


Claude E. Shannon was born in Gaylord, Mich., on 
April 30, 1916. He received the B.S. degree in Elec- 
trical Engineering and Mathematics from the Uni- 
versity of Michigan in 1936. From 1936 to 1940, he 
was at M.I.T., combining graduate studies with 
professional experience. For two years he was a re- 
search assistant in the Electrical Engineering Depart- 
ment, where he operated the Bush mechanical 
differential analyzer. He was an Assistant in the 
Mathematics Department from 1938 to 1940, and 
during 1939-1940, was a Bolles Fellow. He re- 
ceived the S.M. degree in Electrical Engineering 
and the Ph.D. degree in Mathematics from M.1.T. 
in 1940. 

He was associated with the Institute for Advanced 
Study at Princeton University for one year thereafter 
as a National Research Fellow and consultant to the 
National Defense Research Committee. Since 1941, 
he has been a research mathematician for Bell Tele- 
phone Laboratories in Murray Hill, N. J. 

Dr. Shannon’s chief work has been in the follow- 
ing fields: the use of Boolean Algebra in relay and 
switching circuits, theory of communication, Mathe- 
matics of cryptography, theory of differential ana- 


lyzer, and the use of computing machines for non- 
numerical operations. He also has studied chess- 
playing and maze-solving machines, the theory of 
Turing machines, design of reliable machines from 
unreliable components, stochastic processes, the 
Algebra of genetics, and graph theory. 

In 1940, Dr. Shannon was the recipient of the 
Alfred Nobel Prize of the American Institute of 
Electrical Engineers for his work in switching theory. 
He received the Morris Liebmann aiward of the 
Institute of Radio Engineers in 1949 for his com- 
munication theory work. Yale University awarded 
him an honorary Master of Science degree in 1954, 
and in 1955, Dr. Shannon received the Stuart Bal- 
lantine medal of the Franklin Institute for work in 
communication theory. He is the author of approxi- 
mately thirty-five technical papers, and holds several 
patents. He is co-author, with Warren Weaver, of 
“The Mathematical Theory of Communication,”’ 
and co-editor, with John McCarthy, of the forth- 
coming ‘‘Automata Studies.” 

Dr. Shannon is a Fellow of the Institute of Radio 
Engineers, and a member of the American Mathe- 
matics Society, Sigma Xi, and Phi Kappa Phi. 
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The Bandwagon 


CLAUDE E. SHANNON 


NFORMATION theory has, in the last few years, 
if become something of a scientific bandwagon. 

Starting as a technical tool for the communica- 
tion engineer, it has received an extraordinary 
amount of publicity in the popular as well as the 
scientific press. In part, this has been due to connec- 
tions with such fashionable fields as computing ma- 
chines, cybernetics, and automation; and in part, to 
the novelty of its subject matter. As a consequence, 
it has perhaps been ballooned to an importance 
beyond its actual accomplishments. Our fellow scien- 
tists in many different fields, attracted by the fanfare 
and by the new avenues opened to scientific analysis, 
are using these ideas in their own problems. Applica- 
tions are being made to biology, psychology, lin- 
guistics, fundamental physics, economics, the theory 
of organization, and many others. In short, informa- 
tion theory is currently partaking of a somewhat 
heady draught of general popularity. 

Although this wave of popularity is certainly 
pleasant and exciting for those of us working in the 
field, it carries at the same time an element of danger. 
While we feel that information theory is indeed a 
valuable tool in providing fundamental insights into 
the nature ef communication problems and will 
continue to grow in importance, it is certainly no 
panacea for the communication engineer or, a fortiort, 
for anyone else. Seldom do more than a few of 
nature’s secrets give way at one time. It will be all 
too easy for our somewhat artificial prosperity to 
collapse overnight when it is realized that the use of a 
few exciting words like information, entropy, redun- 
dancy, do not solve all our problems. 

What can be done to inject a note of moderation in 
this situation? In the first place, workers in other 
fields should realize that the basic results of the 


subject are aimed in a very specific direction, a 
direction that is not necessarily relevant to such 
fields as psychology, economics, and other social 
sciences. Indeed, the hard core of information theory 
is, essentially, a branch of mathematics, a strictly 
deductive system. A thorough understanding of the 
mathematical foundation and its communication 
application is surely a prerequisite to other applica- 
tions. I personally believe that many of the concepts 
of information theory will prove useful in these other 
fields—and, indeed, some results are already quite 
promising—but the establishing of such applications 
is not a trivial matter of translating words to a new 
domain, but rather the slow tedious process of 
hypothesis and experimental verification. If, for 
example, the human being acts in some situations like 
an ideal decoder, this is an experimental and not a 
mathematical fact, and as such must be tested under 
a wide variety of experimental situations. 

Secondly, we must keep our own house in first class 
order. The subject of information theory has cer- 
tainly been sold, if not oversold. We should now turn 
our attention to the business of research and devel- 
opment at the highest scientific plane we can main- 
tain. Research rather than exposition is the keynote, 
and our critical thresholds should be raised. Authors 
should submit only their best efforts, and these only 
after careful criticism by themselves and their col- 
leagues. A few first rate research papers are preferable 
to a large number that are poorly conceived or half- 
finished. The latter are no credit to their writers and 
a waste of time to their readers. Only by maintaining 
a thoroughly scientific attitude can we achieve real 
progress in communication theory and consolidate 
our present position. 
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The Probability Distribution for the Filtered Output 
of a Multiplier Whose Inputs are Correlated, 
Stationary, Gaussian ‘Time-Series 


D. G. LAMPARD{ 


Summary—In this paper the techniques used by Kac and Siegert 
and by Emerson for evaluating the probability distribution for the 
filtered output of a square-law device with a stationary, Gaussian 
input, have been extended to the case of a multiplier whose inputs 
are a pair of correlated, stationary, Gaussian time-series. It is shown 
that in this case the probability distribution is determined by the 
eigenvalues of a pair of simultaneous, linear, homogeneous, integral 
equations whose kernels involve only the correlation functions of 
the inputs and the impulse response of the postmultiplier filter. 

Explicit solutions for the eigenvalues of these integral equations 
are obtained both for the case of no postmultiplier filtering and for a 
simple example system using RC filters. Using these solutions the 
corresponding probability distributions are discussed and in par- 
ticular, the way in which the probability distribution of the output 
tends to Gaussian as the postmultiplier filter time constant is 
increased, is demonstrated. 


I. INTRODUCTION 


PAIR of stationary time series with Gaussian 
probability distributions are completely de- 
scribed statistically when their cross- and auto- 

correlation functions are specified.” Both for this reason 
and because of the relationship of the correlation functions 
to the corresponding power spectra,’ the technique of 
measurement of such correlation functions has received a 
considerable amount of attention in the literature over 
the past few years.°"* 

If we denote the time series by f,(£) and f2(¢) the cross 
(¢ ~ 7) and auto (¢ = 7) correlation functions y,;;(7) are 
defined by either 


W;;(7) (FOF s(t = T)) (1) 


limp { $OME- Ad, @) 


where in (1) the angular brackets denote ensemble 
averaging. The equivalence of these definitions follows 
from the ergodic hypothesis.” 


+ Commonwealth Scientific and Industrial Research Organization, 
Division of Electrotechnology, University Grounds, Chippendale, 
Sydney, Australia. 

1 Zero means are assumed without loss of generality. 

2 This relationship is generally known as the Wiener-Khintchine 
Theorem. 

3 Fairly complete bibliographies of correlation techniques will be 
found in F. L. Stumpers, “‘A Bibliography of Information Theory,” 
M.I.T. Res. Lab. Elec. Tech. Rep., pp. 11-15; February, 1953 and 
supplement to ‘‘A Bibliography of Information Theory,” N. V. 
Philips Gloeilampenfabrieken, Eindhoven, Netherlands, pp. 8-11; 
April, 1954. 

4D. G. Lampard, “A new method of determining correlation 
functions of stationary time series,’ Proc. THE, Part IV, Mono- 
graph No. 104, 1954. 

®N. Wiener, “Extrapolation, Interpolation and Smoothing of 
Stationary Time Series,” John Wiley & Sons, New York, p. 15; 1950. 


When the f.(¢), f(t) are voltages or currents, a 
analog system of the type shown in Fig. 1 is commonl 
used to measure the correlation function y,2(7). 


MULTIPLIER | 


f(t) 


Fig. 1—Typical analog correlator. | 


In this system a low pass filter of finite time constar 
is used to carry out approximately, the time averagin 
operation implied in (2). It will be clear that the outpu 
fo(t) of such a system is a random function of tim 
whose statistical properties are a function of the tim 
constant of the low-pass filter (that is, for a given typ 
of low-pass filter). Thus the output exhibits fluctuatior 
about its mean y,.(7), the magnitude of which decrease 
as the time constant of the low-pass filter is increase¢ 
While a direct determination of the variance of thes 
fluctuations is fairly straightforward,° the determinatio 
of their probability distribution is a much more difficu 
problem. This difficulty arises because of the non-Gaussia 
distribution of the input to the low pass filter. (Se 
Section ITI). 

From a statistical point of view the essential element 
of the system of Fig. 1 are the multiplier and the filte 
(not necessarily low-pass) and for this reason we sha 
concern ourselves with the system of Fig. 2. 


FILTER 


Fig. 2—Basic system for analysis. 


MULTIPLIER 
x, (t) 


Xo(1 
X2(t) 


For the special case in which the two input time serie 
are identical, 2,(¢) = x(t), this system has been studie 
by several authors, in particular Franz,’ Kac and Sieger 


_ ® Assuming that the fi(¢) and f2(t) are Gaussian. This assumpti 
is made throughout this paper. 

7K. Franz, “Die amplituden von Geranchshanungen”’ Elek. Nac 
Tech., vol. 19, p. 166; September, 1942. The author is grateful 
Mr. R. E. Burgess for bringing this reference to his attention. 

8M. Kae and A. J. F. Siegert, “On the theory of noise in rad 
receivers with square law detectors,” Jour. Appl. Phys., vol. 1 
pp. 383-397; April, 1947. 
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1d Emerson’. These authors have shown that the 
obability distribution of the output «»(¢), is determined 
; the eigenvalues of a linear, homogeneous, integral 
juation whose kernel involves only the autocorrelation 
nction of the input and the impulse response of the 
ter. More recently Siegert’® has pointed out that it is 
ten more convenient to work with a closely related 
homogeneous integral equation. In particular he has 
own that this inhomogeneous integral equation reduces 
a differential equation when the input has been obtained 
y filtering white noise with a finite lumped circuit 
stwork. The way in which the probability distribution 
shaves in the limiting case of a very narrow band filter 
us been discussed by Arthur’* while Meyer and Middle- 
nm” have shown how the treatment of the authors 
entioned above can be extended to obtain multi- 
mensional probability distributions. 

In this paper we shall employ similar methods to study 
le general system of Fig. 2. It will be shown that in this 
ise a pair of simultaneous, linear, homogeneous, integral 
juations arise whose eigenvalues determine the prob- 
nlity distribution of x(t). The special case of no post- 
ultiplier filter will be discussed and finally we shall 
eat in detail a simple example system and show how 
‘plicit solutions for the eigenfunctions and eigenvalues 
the integral equations can be obtained. By making use 
these solutions we demonstrate the way in which the 
itput probability distribution tends to Gaussian as the 
ostmultiplier filter time constant is increased. 


II. ANALYSIS 


The filtered output of the multiplier may be written 
sa convolution integral,” thus: 
t 
Lt) = / x(t — wx(t — u)h(u) du. (3) 
0 
Here the inputs 2,(é), x,(¢) are assumed to be corre- 
ted stationary time series with Gaussian probability 
istributions while h(t) is the impulse response of the post- 
‘ultiplier filter. 
It will be convenient to introduce new variables defined 
y 


X(t) 
X(t) a 


2[ti(t) + 22(t)] 
zlai(t) — 2x2(¢)]. 


(4) 


9R. C. Emerson, ‘First probability densities for receivers with 
uare law detectors,” Jour. Appl. Phys., vol. 24, pp. 1168-1176; 
ptember, 1953. 

WA. J. F. Siegert, “Passage of stationary processes through 
ear and non-linear devices,’”’ Trans. IRE, vol. IT-3, pp. 4-25; 
arch, 1954. 

1G. R. Arthur, “A note on the approach of narrow band noise 
ter a non-linear device to a normal probability density,” Jour. 
ppl. Phys., vol. 23, pp. 1143-1144; October, 1952. 

2M. A. Meyer and D. Middleton, ‘‘On the distribution of signals 
d noise after rectification and filtering,” Jour. Appl. Phys., valx,25, 
. 1037-1052; August, 1954. : 

18 H. S. Carslaw and J. C. Jaeger, “Operational methods in applied 
uthematics,”’ 2nd ed., Oxford University Press, New York, p. 252; 
48, 
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As these new variables are simply linear combinations 
of the 2x,(t) and 2,(¢), it follows, from a well-known 
theorem™ that they also have stationary Gaussian prob- 
ability distributions. 

If further we denote their correlation functions by 


bii(7) = (X(OX,(t + 7), (5) 

then it is easy to show that 
dult) = E{vult) + Yrolr) + vail) + vo2lr)} (6) 
drolt) = 1¥ult) — Piolt) + Walt) — aalz)} (7) 
galt) = £1 ¥ault) + viol) — Walt) — Poal7)$} (8) 
doo(t) = E1¥ilt) — ist) — Wailr) + Yaal7)} (9) 


where y,,;(7) = (x(t) x;(¢ + 7)) are the correlation func- 
tions for the original variables 2,(¢), x(t). 

Then in terms of the new variables X,(¢), X.(t) we may 
write (3) as 


no() = [ {Xt — w = XHt— yh du. (10) 
We shall in fact study the more general form 
no(t) = f {Xie — w & X8E — wjh) dw (11) 
0 


= if Xi(t — whtu) du + ik X3(t + wh(—vw) du. 
(12) 


To make further progress we introduce the discon- 
tinuous functions, 


U, (2) rs i L > 0-- (13) 
Up C0 
whenme te «> 0- (14) 
if EO =s, 
and define new variables, 
Z(u) = Xo sa wu) U,(u) ae X(t PF uU) U,(u) (15) 
Hu) = hw)U(u) + h(—w UU) (16) 
aa h(| U 1) {Ui(u) se UU) }. (17) 
It is clear ‘that 
Zu) = Xt -—wUw) + X2t-wU.w, (18) 
and so (12) can now be written 
Ors / POH du (19) 
= lim: < Ss B24; (20) 


no t=—n,j7=—N 


49. S. Wilks, “Mathematical Statistics,” Princeton University 


Press, p. 70; 1943. 
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where so that (26) may be written 
t +0 +0 
Z, = Lu) = (3 p (21) M(@e) = lim| Af? || if / (2n)"" 
B;; = 1 Hj ‘Nae (22) -exp {—4 X (1 — 210,)y5} dy_, --- dy, (8 


Here 6;; is the Kronecker delta and the dash on the 
summation indicates that the term 7 = 0 is to be excluded. 

As X,(t), X.(t) have stationary Gaussian probability 
distributions, it follows that Z; has a 2n dimensional 
Gaussian distribution. Thus we may write *° 


DL 


a (2m) | A he exp — 2 > > Ais Zi, 


Let, Lind Ae Zin) 


(23) 


where [A] is the inverse of the second moment matrix 
[M] whose typical element 7; is given by 


(Zu) Z(u;)). 


Now let us find the characteristic function M(76) for 
x(t) defined by 


M(i6) = Cy 
lim (2n)~" | A |!” 


no 


M,; = (Z,2;) = (24) 


(25) 


2nfold 


| ae / exp {—3 SY, ~ A;;4;4; 


+ 10%, 3 B,;Z;Z;} dZ_, --- 


As [A] is the coefficient matrix for a Gaussian Multi- 
variate distribution it 1s positive definite and symmetric. 
[B] is not necessarily definite but these conditions are 
sufficient for a simultaneous reduction of the quadratic 
forms, occurring in (26), to principal axes.*® 

Thus there exists a linear transformation 


aL (26) 


+n 


Z; = yy aii (27) 
such that 
bs pe AZZ; = X 9; (28) 
RX dX B22; = XV dy, (29) 
where A; are roots of the equation 
| Bi; as rA,; | = 0. (30) 
The Jacobian of the transformation (27) is just 
CAB why) 
I(Y-ny ae Yn) | | ( ) 
15 §. S. Wilks, loc. cit., 
6R. Courant and Dp. Hilbert, “Methods of Mathematical 


Physics,’”’ Vol. 1, Interscience Publishers, New York, p. 37; 1953. 


= tim | 4 P| 0) EE 


now j=-—n 


[ex [-40 = iar dy 


lim | 4 P| 2] ¥ (l= 21-7... a 1 


j=—n 


However it is clear from (25) that (0) = 1 so that v 
must have | A |? | 2 | = 1 and hence finally we may wrii 


= lim i = 26k) (3 


no 7=—n 


M(i8) 


We now find the cumulants’’ (or semi-invariants) of tl 
probability distribution of x(t). The mth cumulant 

defined as the coefficient of (76)”/m! in the expansion — 
the logarithim of the characteristic function. Thus v 
have 


| 
| 


log {lim J] a- a (3 


[re x} @ 


tee LE on 


—-_ K.., (3 


. (3 


Kq. (39) shows that the cumulants of the probabili 
distribution for x,(¢) are simply proportional to the a 
of the mth powers of the roots of (80). When we use (2 
and (22) we see that (30) may be written 


log M(ié) 


I 


Ms 
cian 
2 
D> 
Nea 


where 


Kp nal) lim SEX 


now j=—n 


t 
| ui) Bai =—N 


that is 


6;; — ~ Hu) M = 0, (4 


where, as before, M;; is an element of the second mome 
matrix defined by (24). 
In the limit, as n >, putting u; = 
becomes 
M;,; = ({Xi(t u)U,(u) + X(t aa u)U.(u)} 
{Xt — YU) + Xo = v)U2(v)}) © 


7H. Cramér, “Mathematical Methods of Statisties,’’ Princet 
University Press, p- 186; 1951. 
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dulu — Uw) Ui) + dru + 2) Ui U2() 
+ da(—v — wU,(u) Ui) 
+ doe(v — u)U2(u) U(r) 


(43) 
= &u, v) (44) 


From the theory of integral equations,** we see at once 
hat (41) is just the Fredholm determinant of the linear 
lomogeneous integral equation 


NO) = [ Bu, NMOS du (45) 


In particular, when a sufficiently long time has elapsed 
or a statistically steady state to be reached, we may 
xtend the limits to infinity obtaining the singular 
ntegral equation 


nfo) = f Bu, NMS 0) ae (46) 


= [Keun st du, (47) 


vhere 
(u,v) = h(| &% |) fou — v)Ui@) U0) 
+ gn(u + Ui) U2() 
+ g2i(—v — u)U2(u)Ui(o) 
| + gn(v — wUU)U2)}. (48) 
f now we set 
fe) = fie)Ui@) + f.(-v) U2), 


ve find that (48) may be written as the pair of simul- 
aneous integral equations 


(49) 


0) = i see ae NORV SE th 
+ | "due — WhW fee) du. 
fa) = | "dno — wh(u) fale) de 


+ | dnnlo — wh falw) du, 
0 
‘inally we write 


fie) = Flqi) + g2(r)} 
fort) = 21g.) — gov)} 


nd consider the plus and minus cases separately. We 
ind on making use of (6), (7), (8) and (9), that (50) may 
ye written as follows: 


(51) 


18 W. V. Lovitt, “Linear Integral Equations,”’ Dover Publications, 
Yew York, p. 24; 1950. 
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if 
Case 1 + *° Filter Input = 4[zi(é) + 23(6)] 
2X¢,0) = ir Yulo — u)h(u)g.(u) du 
+ | vl — whledantod du 
¥ (52) 
2o.(0) = [ Valo — wWh(u)g,(ei) du 
cE a Yoo(v — u)h(u)go(u) du. 
Case 2 — °° Filter Input = 2,(é) x2(¢) 
2dgi(v) = (Pe Violv = wh(u)giu) du 
+ | Yao = whedanto) da 
; (53) 


2NO50) = [ Yo2(v — u)h(u)g,(u) du 


+ | * Yailo — whe galas) dhs. 


Eq. (53) involves only the correlation functions of the 
inputs to the multiplier and the impulse response of the 
postmultiplier filter. 

In particular we note that, if we set x,(t) = x(t), (53) 
reduces immediately to the single integral equation 


Ag) = | ve — whudg(a) du, (54) 
which is identical with that given by Kac and Siegert® 
and Franz’ for the square law detector (after appropriate 
change of notation). 


TII. Tue Case or No PosrMutrieLierR FILTER 


In this section we shall make use of (53) to evaluate 
the probability distribution for the output 2,(t) in the 
special case of no postmultiplier filtering. In this case the 
impulse response h(t) of the filter is just replaced by the 
impulse function 6(¢) and the integrations carried out. 
We find 


giv) = Yrelv)gi(0) + Wii(v) g2(0) 
2drg2(v) = Yoo(v)gi(0) + Yoi(v)g2(0). 


Setting »v = 0 and noting that ¥,,(0) = ¥2(0), we may 
write (55) in the form 


[2 — ¥r2(0)]gi(0) = ¥11(0) 92(0), 


(55) 


(56) 
[21 — ¥12(0)]92(0) = W22(0)g,(0), 
and hence we must have 
[2d mF ¥'12(0)]? a ¥'1(0) ¥22(0), (57) 
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which is the eigenvalue equation whose solutions are 


_ 20) + V 40) 4220), 


d 9 (58) 
Let us write 
yy = ¥20) + Vv) yr0(0) 
y (59) 
nw V W11(0) P22(0) re ¥12(0) 
as 2 
Then it follows from the inequality” 
W11(0) P22(0) hae Y2(0) = 0 (60) 


that both \{, \3 are positive. Then we see from (35) that 


the characteristic function has the simple form 
M(i6) = {(1 — 220n)(1 + 220n5)}7”, (61) 


and the corresponding probability density is given by 
the Fourier transform: 


PENS = | do "(1 — QOnt\(1 + 260n!)}-”2.. (62) 


Fortunately this integral is readily evaluated in closed 
form. 

To carry out this evaluation we first complete the 
square in the denominator of (62) obtaining 


CA nue ie Shey ( + oO 
p(Xo) en (AtAS) Le dé e Ane e 
Urs 


= se 
Ana : 


On changing the variable by the substitution 


=i E ote (63) 


Ux Ne 
Dee Tie 6h 


followed by shifting the contour of integration back to 
the real axis (this step is easily justified) we find 


—_ hg NNR WE ae = (AZ ck A 
P(Xo) Ae (AYA) exp Xo ann 


tai —t£oz M aP my 2 ale 
i. dze {( aN X. +2 : 


Now, Bassett’s integral representation” for the modified 
Bessel function of the second kind and zero order is 


(64) 


se 
é 


+ 2”) 


K(a|2) =5 ip * a (65) 


1/2 dz, 
so using (65) in (64), we have 


19 This follows immediately from the Schwarz inequality applied 
to the temporal definitions of the correlation functions. 

20G. N. Watson, “‘Theory of Bessel Functions,’’ Cambridge Uni- 
versity Press, p. 172; 1948. 


Mare 
1 e a 
eA {- Se 
1/\2 (66 
Neer Mt 
KA Lier 


We now substitute the values given by (59) for Xf, A 
and obtain finally 


pes) = * {Yrr(O) oral) — Y%e(0)}-" 


v2(0) \ 
oe (ns ¥12(0) ¥22(0) — ¥72(0) 


V¥i(0) ¥o2(0) \ . 
Ele oe ee 


This result, which gives the one-dimensional probabilit; 
distribution for the product of two correlated Gaussiai 
random variables, is a special case (n = 1) of a slighth 
more general result given by Wishart and Bartlett,”* an 
is well known in classical statistics. We note also, that in | 
recent interesting paper (which has several points o 
contact with the present work) G. R. Arthur” discusse 
the integral which occurs in (62) and shows that it may b 
evaluated in terms of Whittaker’s confluent hyper 
geometric function. When it is realized, however, that th 
particular hypergeometric function encountered is readil; 
expressible in terms of a Bessel function, some simpli 
fication results. 


IV. Expuicir SoLUTION oF A SIMPLE EXAMPLE SYSTEM 


In this section we shall consider in detail the followin! 
system (see Fig. 3). 


BROADBAND 
NOISE 
(w,/UNIT BANDWIDTH ) 


Fig. 3—Simple example systems. 


The impulse response of the postmultiplier filter is jus 


1 


= fee = he, 
RO) Geog. i> 0 where a yg 0 


(68 


while it is easy to show that the correlation function 
V.;(7) are given by 


1 J. S. Wishart and M. S. Bartlett, ‘‘The distribution of secon 
order moment statistics in a normal system,” Proc. Cambridge Phi 
Soc., vol. 28, p. 455; 1932. 

=G. R. Arthur, “The statistical properties of the output of 
frequency sensitive device,’ Jour. Appl. Phys., vol. 25, p. 118: 
September, 1954. 


Yat) = F 87), all 7 (69) 
V2(7) = me tT>O (70) 
= 0 <0 
volt) = 0, >0 
= 2 ae*", 7 0) me 
Yar) = 356", all r, (72) 


iwhere 6(7) in (69) is the even impulse function and where 
by) 
On making use of (68) through (72), in (53) we obtain 


ug.(v) = ofe ** i! eg, (u) du + Be” go) (73) 
1g.) = Se [ee gw) du 

ark af e*” / en Mat) ida 

+ ape [ O"ga(u) du, (74) 
where we have written 
. 4 
; ee (75) 
| oe 


| We now solve the simultaneous integral equations (73) 
and (74) by reduction to differential equations. This 
technique is essentially that employed by Juncosa”’® to 
solve the single integral equation which arose in an 
example in the work of Kac and Siegert. It should be 
pointed out, however, that the solutions were first obtained 
by the author as a series expansion” in powers of e~”’. 
The series expansion was recognized as that of a Bessel 
function and hence it was realized that a more direct 
approach would be obtained by showing that the solutions 
satisfied Bessel’s differential equation. 

_ We differentiate (74) with respect to v and obtain 


4 » 
Hg20) = asa: eg, (u) du 
2 foe} 
ae oF ee / e S**P4a (yu) du 


+ ape’ | oS" %gx(u) du — aBe*' gale). (76) 


» 


2M. L. Juncosa, “‘An integral equation related to Bessel func- 
tions,’’ Duke Math. Jour., vol. 12, p. 465; September, 1945. 

*% Quite recently an interesting note has appeared on the series 
solution of integral equations. See J. R. M. Radok, ‘“The solution 
of eigenvalue problems of integral equations by power series,” 
Quart. Appl. Math., vol. 12, p. 413; January, 1955. 
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9 
Multiplying (74) by — @ and adding to (76) gives 
ulg22) — age(v)] 
= —a’Be*” / e* "a. (u) du — aBe*’g,(v) (77) 
= —alugivr) — Be?’g.(v)} oa abe *" go(v) (78) 
= —ayg.(r). (79) 


In passing from (77) to (78) we have made use of (73). 
Thus we have 


g2(v) — age(v) = —ag,(v). (80) 
Again, differentiating (77) with respect to v gives 
ulg2’(v) — ag2r)] 
= abe iP eS" “a, (u) du — a’ Be ”’g,(v) 
— ofe "I gs) — Bgov)] (81) 
= a’[ugi) — Be *’go(v)] — a’Be™*"g,() 
— afe (gs) — Bgo(vr)], (82) 


where again use has been made of (73) to remove the 
integral on the right-hand side of (81). 
Now multiply (80) by alu — Be~*’] and add to (82). 
We find on collecting terms and dividing by up, 
gh") + ‘22 


ve (2a — pe” — aha) =0. (83) 


On making the substitutions, 


Lom i2 Eee ailaaK (84) 
gv) = slay" (2 _ 1) exp {2 oht, (85) 


it is easy to show that (83) may be written 
ern) + af") +4 — (22) bie) = 0; 86 


which latter equation will be recognized as Bessel’s 
differential equation of order 2/8, whose solution is 


f(z) = ASoa a(x) + BI cael), ae ~ an integer 
B 
(87) 
= AJoa/e(z) + BY 20/(2), = = an integer 


where A, B are arbitrary constants. 

It will be seen from (74) however, that g.(v) — 0 as 
v —o and hence it is clear that we must have B = 0. 
Thus finally we may write 


2 
g2(0) = Btone = (22 — 1) -exp (Soft, 


(88) 
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with now no restrictions as to whether 2a/8 is an integer 
or not. To find g,(v) we use (88) in (80). Performing the 
differentiation and making use of the recurrence relation 

Ld (2) VIL) = tdi, Se (89) 


we find readily that 


G0) = Ae. = (22 i) exp 2} 


a (2 
eee = (2 > i) exp {8 fh. (90) 


To determine the eigenvalues we evaluate the expression 


Js he ii e* a (u) du + Be” g.(v), (91) 
0 
which is just the right-hand side of (73). To do this, we 
use the series expansions for the Bessel functions occurring 
in (88) and (90), and integrate termwise in (91). 
We find, after some reduction, 


Bo fa (2a _ eh ee 
ea a Jew! S| 
y, 
eee 2 (22 1) 
la [2a 
= PA tase a? 2 (2 a! i)} 


When we note that the first term in this expression 
is just uw g,(v) it is clear that (73) can be satisfied only 
when u takes values such that 


, 
ane ‘ (22 = i)} == 


Remembering, from (75), that » = 4A/w) and writing” 
Yo = (wa)/4 for the total power output of the pre- 
multiplier filter, we see that (93) may be written 


ry | 
BTA ; (% a i)} = 


We have shown earlier in the paper [see (39)] that the 
mth cumulant for the probability distribution of the output 
x(t) may be expressed in terms of sums of the mth powers 
of the eigenvalues \;. Fortunately the computation of 
these sums is very readily carried out when we make use 
of an ingenious technique described by Spiegel’® He has 
shown that, if z; denotes the jth root of the equation 
(whose roots must all be real), 


(92) 


(93) 


(94) 


1+ane + an +a + --- =0, (95) 


25 To show this, set 7 = O in (72). 

26M. R. Spiegel, “The summation of series involving roots of 
transcendental equations and related applications,” Jour. Appl. 
Phys., vol. 24, p. 1103; September, 1953. 


Mare 
then 

y= Fe Oy (9€ 
j=1 U; 
Lo a a — 2a, (9% 
jai Uj; 
2 me = 301A a 3Q3 = (98 
p= 1 7 
ee =a; — daja, + 2a; + danas — dor. (9E 
phil Cae 


Now a theorem”’ due to Lommel states that all the zere 
of J,(z) are real provided v > —1. Thus it follows that 
for 2a/8 — 1 > 0, the only roots of (94) are real an 
hence Spiegel’s method may be applied. Thus, expandini 
(94), we have 


\ 


F-) glee 
ee ea B vi 
PG) GG)" 

o(% BUEN +1 | 

(ea) 

a e ee 0, (104 


ai(22)(72 + 1)(28 oe 2) 2 


and using (39) and (96) through (99) we find the followin 
simple expressions for the first four cumulants: 


Ki = 0 = b (101 
KO (102 
ates ays yale Bogie 
Kita DEE eke ON Sara (103 
Geni UE oy Oi: 
Ste 2 Be Bn gM ees 
epic Ce ereruey aR eT (104 
If we introduce the normalized variable 
t) — kK 
y= 2 
(Ky” io 


then a convenient expansion for the probability distri 
bution of V is given by the Edgeworth series”® 


VAY e "1 OHAV) 


1 
V 20 
a [(C.H.(V) ete C.He(V)] Te +e one (106 


where H,(x) is the Hermite polynomial of degree - 
defined by 
gfe 
dx” 


(oP) = ere (107 


27G. N. Watson, loc. cit., p. 482. 
8H. Cramér, loc. cit., p. 288. 


956 


und where 
a 
EN LTS 4 \B 
Goes CKO) Cae oO, (108) 
8 
a 1 
Keats (2 a i) 
Ca Ke (109) 


An explicit expression for c;, which term in the Edge- 
vorth series is of the same order as the c, term, has not 
een given because expressions corresponding to (99) 
nut for powers greater than four are not yet available.”° 

It is clear from (106), (108) and (109) that for large 
x/8 (the ratio of postmultiplier to premultiplier filter 
ime constants), the distribution of V tends to Gaussian, 
is we might expect from the central limit theorem. We 
ote that other expansions, corresponding to the Edge- 
vorth series, but more useful for intermediate values of 
y/8 have been given by Marcum.”° 

To conclude our discussion of this example we note 
hat for the specific case a/8 = 5/4, (94) may be written 


Jud? Yo, 3) 


When we recall that 


DAE: 
ACh hes Mies sin x, 


t is clear that the eigenvalues are given by 


0. 


(110) 


(111) 


Odea Ae Cuda Duce aoe wl) 


ase “hoa 


We now use (112) in (85) and obtain, for the character- 
stic function 


tis) = TI {1 ~ cay} (113) 
i=l jJ@ 

Thus the probability distribution p(a,) for the output 
o(t) is given by 


—ibzo 


11 - 4% ava} 


The integrand has simple poles on the negative imagin- 
Wy axis only and so evaluating the integral by contour 


1 +0 
Da mee wee 


I 


g=1 


“dd. 


p(xo) = (114) 


? In a private communication Dr. Spiegel advises that he hopes 
t some time to extend his results up to about the 10th power. 
lowever the computations are laborious. 

397T, J. Marcum, “‘A statistical theory of target detection by 
ulsed radar,’’? Mathematical Appendix, Rand Corp. Res. Memo. 
IME. 753, p. 35; 1952. 
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11 
integration we obtain®’ 


s+1_2 


Ss” exp 


=i) ve ash : 
pen) = eer We 


diy K Oe 


P(x) = (115) 


We now introduce the theta function 6, (2; q) defined*? 
by 


6.02; q¢) =1+2 >> (—1)"q" cos 2nz (116) 
n=1 
In particular, setting z = 0 we have: 
60; q¢) =1+2 2) (-1)"¢" (117) 
n=1 


and denoting differentiation with respect to g by a prime 
we find 
040; ) = 2 Ys (- yng” (118) 


Thus finally we may write (115) in the form 


™ ‘et 
Ty, 12yJ 7° 


2 ( 
“oo; exp ‘af ap St > W one) 


= 0, tity KO) 


P(X) = 


which is an explicit expression for the probability dis- 
tributor in this case. 


V. CONCLUSION 


We have shown how the statistical problem considered 
in this paper may be solved in terms of a corresponding 
eigenvalue problem. Explicit solutions of the integral 
equations encountered have been given for a simple 
example. 

We point out finally that a considerable amount of 
time has been spent trying to obtain explicit solutions of 
the integral equations for an example system involving a 
pure delay, as in Fig. 1, but so far little progress has been 
made. Of course the cumulants can be obtained in terms 
of the iterated kernels [derived by successive integration 
using (48)] as shown by Emerson® but the process is 
extremely tedious in practice. 
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Summary—tThis paper is essentially an extension of the optimum 
filtering theory of Zadeh and Ragazzini to the case where the time 
function to be operated upon is available only at a sequence of sam- 
ple instants. As in the latter paper the signal is taken to consist of 
two parts, one being a polynomial p(t) with unknown coefficients 
and known maximum degree n, and the other being a stationary 
random component M(t) with known autocorrelation function. It is 
assumed that the signal has been contaminated before filtering by 
the addition of stationary random noise, also of known autocorrela- 
tion function, and that the input to the filter consists of a sequence of 
impulse functions of constant repetition period 7, each impulse being 
of area equal to the sample value of the signal plus noise at the time 
of occurrence of the impulse. This paper shows how to find the 
weighting function, W(t), of a linear filter which will convert the 
sequence of impulse functions into a smoothed output subject to the 
following conditions: the weighting function W(f) is only nonzero 
over a finite range; in the absence of random components, the inter- 
polation or extrapolation is error-free; in the presence of any random 
signal the approximation at the output follows a least squares law. 
The weighting function of the optimum filter is shown to be piece- 
wise continuous in the intervals {rT < ¢ < (r + 1)T;r = 0, 1, 2, 

- N} and the paper concludes with a discussion of a simple 
example illustrating the practical application of the solution. 


INTRODUCTION 


HE FUNDAMENTAL problem of extracting, in a 
least squares sense, a signal s(t) from a mixture y(t) 
of signal and noise n/(t), 


y(t) = s(t) + nd) (1) 
by means of an optimum linear filter was first formulated 
and solved by Wiener [1] who considered the case where 
the signal and noise are members of stationary random 
processes with known auto- and cross-correlation functions 
and where the function y(t) is available to the filter in the 
form of a continuous waveform. A subsequent paper of 
Zadeh and Ragazzini [2] extended the above theory to 
the case where the signal s(¢) contains, in addition to the 
random component M(t), a polynomial p(t): 


p(t) = s a(f) 


r=0 


(2) 


of known maximum order n, but with unknown coefficients 
{Qo, 1, -** Q,.}, and where the linear filter employed is 
only required to have a finite memory. 

In the practical development of sampled data systems, 
there arise some analogous problems [5-7], with the 
essential difference that information available about the 
input waveform is restricted to its values at discrete 
instants in time. The input waveform to the filter thus 
contains information relating to a periodic or quasi- 


+ Visiting Professor of Electrical Engineering, Columbia Univer- 
sity, New York, N. Y. 


periodic observation procedure on the original continuou! 
waveform s(t), and in practice this waveform approximate 
to a sequence of impulse functions, each impulse having afr 
area equal to the ordinate of the original waveform y(t 
at the instant when the impulse occurs. If the samplin; 
has been performed with a constant repetition period o 
duration 7’, then evidently such a sequence may be writtei 
5 1 
yl) = 2 y(—r2) a(t + 17), (3 
and the problem discussed in this paper is the specificatior 
of the weighting function W(¢) of a linear filter havin; 
the following three fundamental properties: | 

1) In the absence of both random components, th 
filter output is to be just p(t + a@) so that the inpu 
polynomial is reproduced with a time shift, but otherwis' 
without error. If a > 0 the operation is one of extrapola’ 
tion; if a = 0 the operation is instantaneous interpolation 
if a < 0 it is delayed interpolation. 

2) In the presence of one or both of the random com! 
ponents the mean squared error over-all time betwee! 
s(t + a) and the smoothed output of the filter is to be : 
minimum for the weighting function W(¢) satisfyiny 
condition 1. | 

3) The filter is required to have only a finite memory 
implying that the weighting function W(t) shall vanisl 
for all time instants greater than some chosen number 
In the interests of simplicity this number will be take 
to be a multiple of the sampling interval 7 and _ thi 
condition is written | 


W(t) = 0. for 


(SN PDT 


IDEAL 
PREDICTOR 


where JN is an integer. 


M : 
Mt wT Piet 
n(t)e SAMPLING 
SWITCH 


Fig. 1—System flow diagram. 


A schematic representation of the various waveform 
and of the system components pertinent to the followin 
theory is given in Fig. 1. 


1956 


Since methods have already been described in the litera- 
ture for approximating as closely as is desired to weighting 
functions of the type which arise in the succeeding analy- 
sis [3, 4], the specification of the optimum linear filter 
reduces to the specification of an optimum W(t). The 
analytic procedure adopted may be divided briefly into 
three steps: to find the constraint equations imposed by 
the first and third conditions mentioned above; to obtain 
an expression for the mean squared error; and finally to 
find a W(t) minimizing this latter subject to the constraint 
equations. The paper concludes with a discussion of a 
simple example illustrating the procedure. 


CONSTRAINT HQUATIONS 


Since the formulation of the problem has included no 
reference to a specific origin in time, it may be supposed, 
without any loss in generality, that the input to the filter 
in the absence of the random components may be written 
in the form 


(0) 


Di pel) a(t — rT) 


r=—0 


(4) 


and that the filter output is considered at some instant ¢ 
in the interval (0 < t < T). It has been stipulated at the 
outset that the polynomial p(t) is of maximum degree n, 
which means that p(t) can be represented everywhere 
in terms of its values at not more than n + 1 distinct 
points. Arguing from this conclusion, it is evident that 
when the filter is providing an output at time ¢, it must be 
operating on at least n + 1 previous input sample values 
for the output to be certainly free of error. This implies, 
in general, that the weighting function W(¢) must assume 
some values at least over the range [(0 < t < (n + 1)7), 
and in fact we shall assume that W(t) is only required to 
vanish for t > (N + 1)7' where N > n. Since the response 
of a filter with weighting function W(t) to an impulse 
6(t) is just W(t), the response to the input specified in (4), 
may be written 


y pirT) Wit — rP), 


and if the sign of the running index r is changed and this 
expression is equated to the required output p(t + a), 
the implied constraint may be expressed as 
DY w(—rT) W(t + rT) = p(t + a) (5) 
r=0 
for all p(t) of degree n. The previous equation can be 
given a more convenient formulation by the introduction 
of a set of auxiliary functions w(t), ui(t), w(t) ++: un(), 
defined over the interval 0 < ¢ < T and which are related 
to W(t) in the following manner: 


PO Weal): 0 st < Lor = 0) 172302 Ns * (6) 


If these functions are used in (4) and the polynomials 
written in their expanded form by reference to (2), then 
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an equation 


N n n s 
YS Lal) =D eo | 
r=0 s=0 a=0 IV 
results, which must be true for all choices of {ao,@,, +++ a,}. 
This last statement implies that 
N ; t s 
yuo = (EA) @=0,1,2,--9, @ 


and the problem of finding constraints such that, in the 
absence of noise, the filter reproduces exactly the input 
polynomial, has been reduced to the selection of a set of 
functions {uo(t), u(t), wUe(t) un(t)} satisfying the 
nm + 1 equations in (7). It may be noted here that by 
inspection of (7), no general solution exists if N < n, 
confirming the argument given earlier in this section, that 
for error-free reproduction of the input polynomial, the 
filter must be able to remember at least n + 1 previous 
sample values. If N is equal to n, there is, in general, just 
one set of w’s satisfying (7), and these define the weighting 
function with the shortest memory which will accurately 
reproduce the polynomial; the fact that there is only one 
W(t) in this case naturally precludes any optimization 
procedure in the presence of noise. Finally, it will be 
observed that if N > n, there is, in general, an infinity 
of solutions to (7), and the optimization problem will be 
concerned with finding some set of w’s out of this infinity 
of possibilities corresponding to a minimum mean squared 
error in the presence of noise. 


MeraAnN SQuarep HRROoR 


In the case where the composite signal consisting of a 
polynomial p(t) and a stationary random waveform 
M(t) has been modified by the addition of stationary 
random noise n(t), the input to the linear filter may 
be written as 


fo) 


y(t) = , pT) = MGT) =P nel) oe srl), 
and the estimate of s(¢ + a) at the output of the filter 
becomes 


foo) 


s(t +a) = 2) [p(—-rP) 


eM =e nar) aaa 


by a modification of (5). Introducing now the functions 
u,(t) and assuming that they have been constrained to 
satisfy (7) the error e(t) between s,(¢ + a) and s(t + a) 
may be written 


e(f) = s(t + a) — s,(t + a) 


M(t + a) — >> [M(-rT) + n(-rT)\u,(). 


r=0 


(8) 


It may be recalled that the problem was formulated for 
interpolation instants such that 0 < ¢ < T and evaluating 
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the mean squared error over this interval yields, 
7 

(e(t)) = Mt + a) dt 
+ — T >> oa [M(—rT) + n(—rT)] 


7 
uu, dt 


- [M(—sT) + n(—sT)] / 


0 


‘ N 7 


- 3 [Mel oe) i M(t + ou,(t) dt (9) 


If the above averaging procedure is repeated over all 
intervals of length T such that [rT < t < (r — 1)7], 
where 7 ranges from —o to +o and an average taken 
of all these results, then this is evidently just the same 
as performing one averaging operation over the range 
(—© <t< o~). Witha little algebra it may be shown that 
this result is 


Cex(h)) et ey) + >> oI oul — s)T] 
+ ¢,[(7 — s)T]} ie uu, at 


70 


7 se i ee nn ce 


pe ANG) 


(10) 


where ¢,;(7) 1s the autocorrelation function of the signal 
process M(t); ¢,(7) is the autocorrelation function of the 
noise process n(t); and (M*(t)) is the mean power of the 
waveform M(t). In deriving the above expression it has 
been assumed that both of the random functions M/(¢) and 
n(t) have zero means and that they are linearly inde- 
pendent. Otherwise additional terms appear in the 
resulting equation for the mean squared error. 


MINIMIZATION PROCEDURE 


On the basis of the preceding formulation the problem 
remaining is to choose the set of functions {w,} to be that 
particular set satisfying the constraints of (7) which 
minimizes the right hand side of (10). For convenience 
and brevity it will be assumed in the following analysis 
that the sampling repetition interval 7 is unity. This 
involves no loss of generality since the optimum W(t) 
obtained on this assumption may be generalized by 
replacing ¢ by t/7' and on the basis of this assumption the 
equations relevant to the minimization problem may 
now be summarized as, 


(e(t))0 = (M"(2)) 
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The constraint equations have to be true for all ¢ in 
(0 <¢ < 1) and since the right-hand sides of these equa- 
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tions are functions of time, then on introducing the | 
\ 


undetermined multipliers {Xo, 44, °-* , A,} which will be 
themselves functions of time, and upon noting that 
(M’(t)) is a constant, the problem is transformed into the 
minimization of the expression, 
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To obtain a more convenient notation define the functions 
y,. and 6,(t) as 


Vis = ou(r — 8s) + o,(7 — 8) 
6,(t) = 2ou(t + a+r), 


(14) 
(15) 


and now perturbing the {u,} by the addition of arbitrary 


functions £,(¢) in the interval (0 < ¢ < 1), the variation © 


of (18) becomes 


ya p> DS v(t) = > MO(—s)" — ao} dt 


and this will be always zero for all choices of {£,}, provided 
that the {u,(é)} satisfy the N + 1 simultaneous equations 


N 


x WrsUr(t) = 8,(t) + dX (—s)'d, (0). (16) 
= =0 
If y,, is a typical element of the inverse of the matrix [y], 


then the solution for u, may be written explicitly as 
00) 


and all that remains is to evaluate the undetermined 
multipliers from the n + 1 constraint equations (12); thus 
the ,(t) satisfy 
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where the element of the inverse matrix of [U] is 


w= DL 2 (—-7)'(— mM)". 


r=0 m=0 


(19) 


Thus using (18) the multipliers \, have the explicit 
representation 
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and this together with (17) provides the complete ana- 
lytical solution to the problem of specifying the optimum 
set {u,} and thus to the problem of specifying the weight- 
ing function W(t) of the optimum linear filter. Provided 
that the functions 6,(¢) are continuous in the interval 
(0 < t < 1), which from a physical point of view seems 
reasonable, then the members of the set {u,} will be con- 
tinuous and thus W(t) itself is piecewise continuous in 
the intervals: {0, 1}, {1, 2} --- {N, N +; 1}. 

In certain special cases of practical importance the 
solution obtained above assumes a simpler form and 
consideration will now be given to two such cases. The 
input signal s(¢) was assumed in the beginning to consist 
of two parts M(é) and p(t) and let attention be restricted 
now to the case where M(t) is identically equal to zero. 
Then following through the arguments of the preceding 
sections it is evident that the constraint equations, which 
do not depend on the presence or absence of an M(t), 
will be unchanged, and thus the mean squared error over 
the infinite interval (—° < ¢t < o) has the form 


EO = Se >» or — 8) i u,(t)u,(t) dt. (21) 


From this we deduce immediately that the 0@,’s can be 
made zero in the final solutions in (17) and (20) yielding 


uf) = Lv D(-9 MO © =0,1,---N), 22) 
eae aU lt0)° ee AOsteete a3). HOD) 
where in this case 

IMO Oe (24) 


Combining (22) and (23) gives an expression for u,(t), 


nt) = Dv} 9)" Ye Umalé + a)" 
= DK alt +9)", (25) 
where the matrix K,,, is defined by 
Ke = > (-9)'U ave 26) 
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It may be observed that if (26) is multiplied by (—r)” and 
summed over r from 0 to N, then on using (19), an equation 
N n 
dy ane = 2» OO (27) 
results. Now if the summation over k on the right-hand 
side had been from 0 to N instead of from 0 to n, then 
this would have yielded the Kronecker symbol 6,,, indi- 
cating that [K] has to be the matrix inverse to [(—r)’]. 
As would be anticipated this is just the solution for the 
u,(t) in the case where n = N and only one set {w,} exists 
satisfying the constraint equations. In the general case 
the right-hand side of (27) defines a nondiagonal matrix 
depending on the correlation matrix [y] and the solution 
is no longer so simple to obtain. It may be noted finally 
that in this special case where M(t) < 0, the solution in 
(25) defines an optimum weighting function W(¢) which 
is plecewise continuous in the intervals (0, 1), (1, 2) --- 
(N, N + 1) in the form of polynomials of degree n. 
The second special case of the general solution, (17) 
and (20), which will be considered, is where the noise 
amplitudes at successive sampling instants are linearly 
independent, and where it is assumed again that M(t) A 0. 
In this case the correlation matrix [¢,(r — s)] of the noise 
amplitudes is diagonal; 7.e., 
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where o° is the noise power. If this result is used in (22) 
then the expression for u,(t) may be written 
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where now, the X,(¢) satisfy equations 
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Defining the matrix Q,, by 


N 
(pa retin (: =r). aoe n), (31) 
r=0 
the solution for ),,(¢) becomes, from (30) 
1) =o >> Q(t + a)’. (32) 
s=0 


This last summation may be computed, using (31), to 
find the numerical values of the coefficients in the poly- 
nomials {w,(¢)}. 


EXAMPLE 


As an example of the use of the analytical result derived 
above, consider the case of a linear predictor with the 
following special conditions; M(t) = 0; m = 1; and with 
the noise uncorrelated between sampling instants. 
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Then from (31) the matrix [Q]"" has the form 
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and upon inversion of this, the matrix [2] may be written 
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Using this result in (32) yields 


[Q] = (34) 
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For the special case of N = 2; 
5 r 1 ik 
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and the functions become explicitly 
Distinct 
ug B= 6 + 9 (t + a) 
w(t) = 3 (37) 
1 1 


Usb) = Se (t+ a). 

The corresponding weighting function is shown in Fig. 
2(b) and may be compared with the form of the “shortest— 
memory” weighting function Fig. 2(a). For the special 
case N = 3, a similar procedure yields the set {w,(d)} 


7 3r 3 ip 
u(t) = 75 —~qg t E+ aye a . (38) 
or more explicitly, [Fig. 2(¢)] 
a 3 
Uo(t) — Io + i109 6 + 
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ult) = 79 — Io (+a) 
2 3 
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It is of interest to compare the mean squared error in 
each of these two cases with that for the case of N = 1. 
In this latter case the w,(¢) are 


u(t) = 1+ + 2) 
u(t) = —( +a), 


(40) 
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using (12); and substituting these expressions in the 
equation for the mean squared error 


CW))o = 0 ey, ue(t) dt, 


which is a modification of (21), using condition (28), the ° 


result becomes 
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Fig. 2—Weighting functions straight-line prediction with 


In the case N = 2, an analogous calculation shows that 
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and for N = 3, 
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ividently the mean squared error is reduced for all 
values of a, by the successive increases in the value of N; 
n fact, for instantaneous interpolation, where a = 0, the 
nean squared error is reduced by factors 0.5625 and 0.4 
rom its value at N = 1, for values of N equal to 2 and 3 
espectively. 


CONCLUSION 


It will have been observed from the nature of the ana- 
ytical solution to the optimization problem, that the 
veighting function W(t) is finite and piecewise continuous 
ver its range of definition. Unlike the optimum weighting 
unctions of Zadeh and Ragazzini, it does not involve any 
mpulse functions. The presence of such singularities 
vould imply a direct transmission through the filter (with 
1 time shift) of the input waveform and since this latter 
sonsists of impulse functions, then the filter output itself 
vould contain impulse functions. It was taken to be axio- 
natic that in a smoothed output, singularity functions 
vould be undesirable. On the other hand the optimum 
veighting function contains, in general, discontinuous 
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jumps at the points 0, T, 27, --- , (NV + 1)T and this 
implies that the synthesis should be active. To find an 
optimum W(t) which is continuous in value over the 
range [0 < t < (N + 1)T] would require the introduction 
of (NV — 1) further constraints. 
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The Third London Symposium on Information Theory’ 


NELSON M. BLACHMANT 


T THE FIRST London Symposium on Informa- 
A tion Theory, held at the Royal Society in Sep- 

tember, 1950, about twenty papers were presented, 
mcluding six on applications of information theory to 
psychology and neurophysiology. At the second sympo- 
sium, held at the Institution of Electrical Engineers in 
September, 1952, emphasis on the latter fields was replaced 
oy an.emphasis on the transmission and analysis of speech, 
vith which eight of the thirty-eight papers dealt. The 
scope of the third London Symposium on Information, 
1eld at the Royal Institution in September, 1955, was 
sroadened to include not only all of these fields but the 
mechanical translation of languages as well, in addition 
0 the basic topics of information theory, coding, etc. 
[he breadth of its scope is also indicated by the range of 
yackgrounds of the participants, which included: anatomy, 
inimal welfare, anthropology, computers, economics, 
lectronics, linguistics, mathematics, neuropsychiatry, 


* The author attended this symposium under the auspices of the 
J. S. Army Signal Corps. a ie : 
+ Electronic Defense Lab., Mountain View, Calif. 


neurophysiology, philosophy, phonetics, physics, political 
theory, psychology, and statistics. Of the 250 partici- 
pants, half were British; one-eighth came from the United 
States (Shannon was not among them); the remainder, in 
order of decreasing numbers, were from the Netherlands, 
Sweden, France, Germany, Denmark, the USSR, Italy, 
Belgium, Switzerland, Spain, and Israel. 

The auditorium of the Royal Institution, at 21 Albe- 
marle Street, between Berkeley Square and Picadilly 
Circus, suited the size and type of the meeting perfectly, 
being semicircular in shape, with steeply rising tiers of 
seats, so that everyone in the room could see and hear 
everyone else. The program of the symposium was 
arranged to cover five days, Monday to Friday, September 
12 to 16, with about seven papers presented each day; 
three in the morning, with a 15-minute coffee interval 
between the second and third; and four in the afternoon, 
with a half-hour tea interval in the middle. In this way, 
there was always a break just before or Just after each 
paper, and possibility of tedium was easily avoided. 
Forty-five minutes were allotted to each paper, of which 
fifteen minutes were for the presentation and the remainder 
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for discussion which generally proved quite lively. This 
arrangement, coupled with the prior distribution of mimeo- 
graphed copies of all papers to the participants, proved 
quite felicitous, and Professor E. C. Cherry of the Imperial 
College of Science and Technology, who organized the 
symposium, deserves to be complimented for his efforts. 


Monpbay: INFORMATION THEORY 


Professor B. van der Pol, Director of the CCIR (Inter- 
national Radio Consultative Committee), Geneva, very 
ably chaired the first day’s sessions of the symposium. He 
started it off with a reference to his own studies of the 
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The next paper, ‘“T'wo-dimensional Photographic Auto- 
correlation of Pictures and Alphabet Letters,” by W. 
Meyer-Eppler and G. Darius of the Institut fuer Phonetik 
und Kommunikationsforschung der Universitaet Bonn, 
dealt with a very interesting and simple mechanism for 
the display of the cross-correlograms of pairs of pictures. 
One transparency, backed by ground glass, is used as a 
source of illumination,’the light passes through a smaller 
transparency at some distance, and the correlogram is 
produced on a ground-glass screen at an appropriate 
distance beyond. The paper discussed the symmetry 
characteristics of the correlograms and their possible 
application to a reading machine. 


effect of successive translation, in particular from English 
to French to English to French to English, each time by 
a different translator. He found that the meaning of the 
original text was retained through the translations but 
the style of the writing was lost, being replaced to a 
certain extent by the translators’ style. 

The first talk on the program was that of Professor 
J. L. van Soest of the Hague on “Some Consequences of 
the Finiteness of Information.’ He discussed ways of 
ensuring that the entropies of continuous distributions 
be non-negative. 

He was followed by G. Spencer Brown of Christ Church 
College, Oxford University, who talked on “Chance and 
Control: Some Implications of Randomization.” Brown 
called attention to some difficulties in the generation and 
testing of random sequences. He also pointed out that in 
the experimental testing of hypotheses, the statistical 
significance of the result is not a measure of the amount 
of information obtained. A heated discussion followed 
between Brown and I. J. Good of Cheltenham, Good 
claiming that a finite sequence must be regarded as 
random if the hypothesis is not rejected that it is part of 
an infinite random sequence. Good also pointed out that 
to every real number x between 0 and 1 corresponds an 
infinite sequence, e.g., the decimal or binary expansion of 2, 
and that for nearly all x this sequence is random. 

The third paper, “On Some Measures of Information 
Used in Statistics,” by M. P. Schuetzenberger of Paris, 
used lattice theory to discuss measures of information. 

The afternoon session of the first day began with 
“Optical Transmission,’ by D. Gabor of the Imperial 
College of Science and Technology, London. Dr. Gabor 
discussed the transmission of light from one aperture 
(the input) to another aperture (the output). Here 
Shannon’s 2W7-dimensional vector describing a signal 
voltage is replaced by a Hermitian illumination matrix. 
The diagonalization of this matrix corresponds to the 
resolution of the optical field into its separate incoherent 
components. Each such component corresponds to one of 
Shannon’s vectors. In the astronomical case, the focusing 
of a telescope at infinity effects this diagonalization, 
resolving the field into its individual incoherent point 
sources, the stars. 


J. F. Schouten of the Philips Telecommunication Indus- 
tries, Hilversum, the Netherlands, then presented a paper 
on “Tgnorance, Knowledge, and Information,’ which 
discussed the concept of “irrelevant information” and 
suggested that the sum of the measures of ignorance and 
information should be a constant during any communica- 
tion process. In the discussion that followed, I. J. Good 
suggested that the random variable X be distinguished 
from x, any value which this variable may take on, in 
order to distinguish averaged conditional entropies from 
those which have not been averaged over X. He also 
suggested a notation for the amount of information about | 
X which is contained in Y when Z is known to equal gz, ' 
viz., I(X:Y/z). Yehoshua Bar-Hillel of M.I.T. took! 
exception to an example Schouten had given in which | 
information had reduced knowledge and increased igno- | 
rance. He objected to this paradoxical use of familiar | 
terms, and he pleaded, as he was to do several times’ 
during the symposium, for ‘‘semantic hygiene.’’ Schouten’s | 
situation was one, he said, of “‘less certainty,’ rather than ° 
“less knowledge.”’ 


The last talk on the first day, a very interesting one, | 
dealt with “Strategic Information in Games and in, 
Voting,” by Robin Farquharson of Nuffield College, . 
Oxford. Farquharson discussed the difficulties which can 
result from the voter’s inability to express all of his, 
preferences in the casting of a single vote. Learning ; 
afterward how the others voted, one may consequently , 
wish one had voted differently. The situation in which no 
one would want to change his vote after finding out how | 
the others voted is called equilibrium. According to game 
theory, whenever voting is open, an equilibrium point 
exists such that no voter need regret his choice of voting . 
strategy. However, perfect information about others’ | 
votes makes possible the bandwagon and underdog effects, ! 
which may yield equilibria without giving the voters their . 
first choice that might also have been an equilibrium. A 
secret ballot is therefore necessary in order that voters 
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may have strategies which are unconditionally best, for, | 


if other voters are free to respond either way to one’s own 
vote, then, whatever strategy one may choose, one may . 
prove to have made the wrong choice. 
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Professor Arthur E. Laemmel of the Polytechnic 
Institute of Brooklyn started off the second day of the 
symposium with a talk on ‘A General Class of Codes and 
their Physical Realization,’ emphasizing the “general- 
purpose” realization of coding through the use of a 
multiple-track magnetic drum on which are recorded 
pulses that control the sequence of coding operations. 


He was followed by Peter Elias of M.I.T., who talked 
on “Coding for Two Noisy Channels,’ namely, the 
“binary symmetric channel,” in which each of the two 
possible transmitted symbols has the same probability of 
being received correctly and the complementary prob- 
ability of bemg mistaken for the other, and the “binary 
erasure channel,” which is similar except that errors are 
received as blanks (a third symbol) rather than as the 
incorrect symbol. Elias has found that when transmitting 
at rates much below the channel capacity, it pays to be 
clever about coding rather than to use Shannon’s random 
code. In particular, he has found that check-symbol codes 
are as good as any other kind, in terms of both maximum 
transmission rate and error probability. 

Elias has not studied particular codes, but David 
Slepian of the Bell Telephone Laboratories has studied 
certain codes for the symmetric binary channel in con- 
siderable detail. Although it was not on the program of 
the symposium, his very interesting paper ‘‘A Class of 
Binary Signaling Alphabets” will evidently be incorporated 
into the proceedings. Slepian starts with an ‘“alphabet’’ 
of 2° n-binary digit “letters’’ which form a group under 
addition modulo 2. With each of these letters he associates 
2”" different n-binary-digit symbols which are at least 
as close to it (in the agreement of digits) as they are to 
any other letter. Each associated set is obtained as a 
coset of the corresponding letter. In this way, he is able 
to construct a maximum-likelihood detector. The prob- 
ability of correct detection can be maximized by an 
appropriate choice of the alphabet, for the determination 
of which Slepian has compiled an extensive table. In this 
alphabet, there are k binary digits, fixed in position, giving 
information, and n — k parity-check digits, which are the 
sums modulo 2 of fixed subsets of the information digits. 

D. A. Huffman of M.I.T. was next on the program with 
an excellent paper on ‘“The Synthesis of Linear Sequential 
Coding Networks” out of modulo-2 adders and delays of 
fixed size. Because of the possibility of having feedback 
paths within such networks, their outputs are, in effect, 
parity checks over an infinite past range of network inputs. 
Their transfer characteristics may be described by rational 
functions of the delay operator D. Huffman has studied the 
meaning of the “natural frequency” and the steady-state 
and transient responses of such networks and has developed 
methods for their realization using the least possible 
number of delay elements. When such a network incor- 
porates feedback, it is capable of delivering an output 
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with no input (7.e., a sequence of zeros as input). Such an 
output, which is necessarily periodic, is called a “null 
sequence” of the inverse network, since the network 
having the reciprocal transfer characteristic would yield 
a sequence of zeros as output with this null sequence as 
input. When its transfer characteristic is a polynomial of 
degree n in D, the longest possible null-sequence length is 
2" — 1. Such a maximum-length null sequence, interest- 
ingly, has properties somewhat like those of random noise. 

Next came “An Elementary Proof of a Special Case of 
the Coding Theorem” by Professor G. A. Barnard of 
Imperial College, London, who proved the coding theorem 
for the symmetric binary channel. During the discussion 
period following Barnard’s paper, S. K. Zaremba of Boulton 
Paul Aircraft Limited, Wolverhampton, introduced his 
own ‘Note on the Fundamental Theorem for a Discrete 
Channel with Noise,” which asserts, on what appear to be 
dubious grounds, that Shannon’s proof of this theorem is 
not correct. 

Dr. Lars-Henning Zetterberg of Stockholm then talked 
on “A Comparative Study of Delta and Pulse-Code 
Modulation.” ‘Delta modulation” involves the trans- 
mission of either a positive or a negative pulse (of the 
same size) at regular intervals, accordingly as the sum of 
all the pulses previously transmitted is less or greater 
than the present level of the modulating signal. Zetterberg 
compared single-channel delta and pulse-code modulation 
from the point of view of information theory and of the 
quality of transmission of speech-like signals in the 
absence of noise. He found that both systems have about 
the same channel capacity when their numbers of ampli- 
tude levels are the same and are large. When parameters 
are chosen to give a maximum ratio of signal to overloading 
and granulation noise, he found that delta modulation 
needs a much larger bandwidth than PCM if the desired 
ratio is moderate or high, but it becomes relatively more 
favorable when the bandwidth of the speech signals is large. 

Next Mr. R. A. Fairthorne of the Royal Aircraft 
Establishment, South Farnborough, talked on “Some 
Clerical Operations on Languages,” which dealt with 
marshaling as well. He was followed by Calvin N. Mooers 
of the Zator Company, Boston, who discussed “Informa- 
tion Retrieval upon Structured Content” for the catalog- 
ing of documents. He suggested the use of interlocking 
sets of descriptors, to indicate the interrelation of the 
descriptors, each of these sets possibly being designated by 
the superposition of corresponding random codes. The 
practicability of such methods, however, has not yet 
been ascertained. 


WEDNESDAY: LANGUAGE ANALYSIS AND MECHANICAL 
TRANSLATION 


The third day began with ‘Negative Entropy of Welsh 
Words,” by Dr. D. A. Bell and Professor Alan 8. C. Ross 
of the University of Birmingham. This paper ostensibly 
concerned the redundancy in the formation of English 
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words of n letters and of Welsh words of n phonemes. In 
both cases, the redundancies were determined by counting 
n-letter or n-phoneme words in a dictionary. In the case 
of English, the results were then weighted according to 
the frequency of n-letter words, and a mean redundancy 
per letter of English was thereby determined. Since a list 
of word frequencies for Welsh was unavailable, it was not 
possible to commit this same error for Welsh. During the 
discussion period, quite a number of objections were raised. 

The next paper was “Mathematical Theory of Word 
Formation,’ by Professor W. Fucks of the Physikalisches 
Institut, Technische Hochschule, Aachen. He discussed 
the empirical distribution of the number of syllables per 
word in English, Arabic, German, Esperanto, Greek, 
Japanese, Russian, Turkish, and Latin, to which he was 
able to fit Poisson distributions shifted one unit upward 
so that no word has less than one syllable. During the 
discussion period the point was raised that Russian has 
several words of no syllables. Fucks had no theory to 
explain his success with Poisson distributions, but he 
exhibited a sort of pin-ball machine which is able to 
approximate this distribution. 

Professor Benoit Mandelbrot of the Universities of 
Geneva and Grenoble then discussed ‘‘“Two Statistical 
Laws of Language,’’ which included ‘A Law of Berry 
and the Definition of Stress,” dealing with the location of 
the syllabic and word stresses, and “A Thermodynamical 
Study of Systems of Categories with Willis (Natural) 
Structures.’ The Willis distribution, which states that the 
proportion of objects falling within the n-th largest 
category is bn “"*®, with a between 0 and 1, according to 
Mandelbrot, can be considered as an approximation to an 
exceptional stable distribution of Cauchy-Paul Lévy. His 
study is said to parallel that of the normal stable distri- 
bution of Laplace-Gauss that arises in thermodynamics. 
During the discussion period, Professor A. 8. C. Ross, a 
linguistician, indicated his own disagreement with the 
Zipfian law of word frequencies, which asserts the applic- 
ability of the Willis distribution. Marcel Golay of SCEL 
and Professor van der Pol suggested the information- 
theoretical study of music. It was then reported that 
random music in the styles of Bach and Vivaldi has 
already been generated at the Bell Telephone Labora- 
tories. Henry Quastler of the University of Illinois claimed 
that Texas cowboy songs can be generated by a first-order 
Markoff process, 7.e., by a random process that depends 
upon only the previous note, supplemented by words 
chosen randomly according to certain frequencies from 
an appropriate list. 

The next paper, by Professors 8. Ceccato and E. Maretti 
of Milan, was titled “The Construction of a Translating 
Machine.” This paper dealt with the difficulties confront- 
ing the designer of a translating machine, particularly if 
he wants to be sure that ideas get across correctly. Nota- 
tion was presented which may have been intended to assist 
in the translation of ideas, but, perhaps because no 
mechanical apparatus is presently available to translate 
Italian into English, its significance was lost. Afterward, 
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Bar-Hillel suggested it is hopeless to try to translate 
“ideas” rather than language symbols, since authors’ 
ideas are often hard to pin down. 

Two somewhat related papers were then presented 
together: ‘Influence of Context upon Translation,” by Dr. 
A. D. Booth of Birkbeck College, London, and “A Program 
for Braille Translation,’ by J. P. Cleave, which dealt with 
the programming for an automatic digital computer— 
presumably the AP#(X)C—of the transcription of English 
into Braille. This problem is of interest because Braille 
uses a number of contractions in ways that are a little 
complicated to describe; 7.e., the transcription is affected 
by the context. Dr. Booth demonstrated that only 14 
comparisons are necessary to locate a given word in the 
Concise Oxford Dictionary; binary searching through a 
translating-machine’s dictionary should thus be much 


faster than comparing with all entries in sequence. In the © 


discussion period, Professor van der Pol pointed out a 
very interesting example of the influence of context upon 
translation—a nearby street sign reading ST. GEO. ST. 


Fairthorne suggested that the process of translation, — 
translation back, correction of errors, retranslation, ete., 
until no errors remained would resemble the search for 


the eigenvalues of a matrix (or operator). 

Following the afternoon tea interval, Professor Victor 
Ynegve of M.I.T. talked on “The Translation of Languages 
by Machine.” At M.I.T. progress is slowly being made on 
the mechanical translation of German into English; 
machine dictionaries still need to be set up. The machine, 
knowing the grammars of the two languages and thus 
being able to recognize types of words, such as parts of 


speech, is to code the input message into a transition | 


(intermediate) language having little redundancy, then 
decode it into the target language subject to the constraints 


of the latter. This intermediate language might be useful — 


in a secrecy system. Margaret Meade suggested, during 
the discussion period, that pidgin English, whose vocabu- 
lary is 75 per cent English and whose grammar is general- 
ized Melanesian, might be a good language to work on, 


since it exhibits the important problems of mechanical | 


translation. 

The last talk of the day was a very lucid one on ‘‘Experi- 
ments in Mechanical Speech Recognition,”’ by Drs. D. B. 
Fry and P. Denes of University College, London. Their 
recognizer uses about twenty 4-octave filters in the 


range from 125 eps to 8 ke, computes the matrix of cross- | 
products of pairs of filter outputs, and determines the 


most likely of 15 sounds on the basis of the largest cross- 
product, taking into account the probabilities determined 


by the preceding phoneme. The recognizer types out the — 
phonemes it recognizes, including spaces. It does fairly well 
with t, k, l, m, n, s, sh, 7, a, u, and uh, but was not able to — 
recognize p, f, th, or r. It was found that, while the use ~ 


of a posterior? probabilities was generally quite helpful, 


it sometimes resulted in the loss of large parts of words. — 
The machine’s performance dropped 50 per cent when it 
listened to a speaker different from the one for whom it 


had been adjusted. 
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THurspAY: MEANING AND THE HuMAN SENSES 


Thursday’s session began with ‘The Place of ‘Meaning’ 
1 the Theory of Information,” by Dr. D. M. Mackay of 
ings College, London. Mackay defined the “meaning” 
f a message in terms of the change it produces in the 
scipient’s ‘conditional probability matrix’ (array of 
robabilities of anticipated events). In response to a 
uestion about “‘aesthetic information” and the ““meaning”’ 
f music, Mackay indicated he was willing to include the 
fects on the recipient’s internal secretions, etc. Bar- 
fillel stated that he had a hard time getting anything 
ut of Mackay’s talk and felt it was pointless to define 
meaning’ in terms of even vaguer matrices. A. 8. C. 
oss also questioned Mackay’s ideas. 

At this point the Russian delegation finally arrived at 
1e symposium. There were three fairly young men, 
vidently all from the Post-Telephone-Telegraph Labora- 
ory in Moscow, and a girl whose job was translator. 
ubsequent investigation over beer mugs revealed that 
1ey were cordial, but their evident unfamiliarity with 
ooken Western languages prevented any real communica- 
on with them. 

Professor V. Somenzi of the Istituto di Fisica, Rome, 
1en delivered his talk ‘‘Can Induction be Mechanized?” 
f amounted to the claim that if the methods of induction 
an be described precisely, then they can be programmed 
9 an automatic machine. Somenzi does not distinguish 
etween extrapolation and induction, asserting it is only 
ecessary to decide which of several possible laws of 
<trapolation fits best. In the discussion period, Mackay 
aimed his own prior invention of an inducing machine, 
nd someone else pointed out that it has long been known 
1at any precisely describable operation can in principle 
e mechanized. 

Next, P. Marcou and J. Daguet of the French Centre 
‘ational d’Etudes des Télécommunications discussed 
New Methods of Speech Transmission”. The method 
volves feeding the speech wave into a single-sideband 
ippressed-carrier modulator, passing the output through 
limiter, and feeding this constant-amplitude signal to a 
equency divider. In this way, it is possible to compress 
1e bandwidth of the speech by a factor of as much as 
28 when the divider divides by this number. The speech 
ave is effectively recovered by multiplying the frequency 
ack to its original value and synchronously demodulating 
. The resulting output is distinctly different in form from 
le original speech wave, on account of the limiter, and 

known as ‘‘constant-level speech,” but its spectrum 
sembles that of the original speech wave, and it sounds 
sry much like the original, although no explanation has 
en found for the latter phenomenon. Similar results 
wve been obtained by digitalizing the system and replac- 

g the speech wave by pulses occurring at its zero crossings. 

This constant-level speech makes possible the use of a 
ass-C rf system, and it yields superior intelligibility in the 
esence of noise. However, the reduction in redundancy 
fected by bandwidth compression renders the system 
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susceptible to bad distortion as a result of slight malad- 
justments. Marcou and Daguet regard the original speech 
wave as s(t) = a(t) cos ¢(¢), where a(t) is its “instantaneous 
amplitude” and d¢/dt is its “anstantaneous frequency,” 
which may in fact be either positive or negative. A 90- 
degree phase shift of all of the Fourier components of 
s(t) yields its Hilbert transform S(t) = a(é) sin ¢(d), which 
would make possible the separation of s(¢) into a(t) and 
¢(t), cos ¢(t) being the output of the synchronous detector 
in the proposed scheme. Unfortunately, the Hilbert 
transform of s(¢) depends upon all values of s(é), past, 
present, and future; hence, such a decomposition of s(¢) 
is not possible while its future is unknown, except perhaps 
when it is quasi-sinusoidal. Thus, it is not clear what the 
system does nor how it works, but the fact that it does 
work is certainly noteworthy. In the discussion period, 
perhaps as a result, a number of people expressed confusion 
as to the nature of the system. J. C. R. Licklider of M.I.T. 
reported that he had listened to the system and that it 
indeed sounded very good—much better than ordinary 
clipped speech. E. C. Cherry described this paper as 
representing good experimental work but lacking in 
theory; the system preserves the intelligibility of speech, 
but we don’t even know, he said, what it is about speech 
that it preserves. 

D. J. Bruce of the University of Reading Psychology 
Department then spoke on ‘“The Effects of Context on the 
Intelligibility of Heard Speech” as measured in the 
presence of thermal noise. By ‘‘context’’? he meant the 
general nature of the words in an unconnected list, such 
as “things to eat.’’ He found that guessing the context 
correctly is helpful, but guessing it incorrectly can result 
in an extremely low score. 

Next, Professor J. C. R. Licklider of M.I.T. spoke on 
“Auditory Frequency Analysis.”” He discussed the in- 
compatability of the place and frequency theories of pitch, 
and he discussed Huggins’s demonstration of the appear- 
ance of a pitch when white noise is fed to both ears, arriving 
at one ear through an all-pass filter; the pitch that is 
heard is related to a frequency characteristic of the filter, 
although neither ear alone detects any pitch. This phe- 
nomenon requires phase or time analysis and cannot be 
explained by strict place theory; 7.e., by a theory which 
postulates that every pitch is associated with a particular 
“place” in the cochlea. By means of a tape recording, 
Licklider demonstrated that the low pitch associated with 
a set of three or so of its neighboring higher harmonics 
can be detected even in the presence of low-pitched thermal 
noise that completely masks an equally loud fundamental. 
He concludes that the ears perform running analyses of 
three types: (1) spectrum, (2) autocorrelogram, and (3) 
cross-correlogram, and that through a process of associa- 
tion of nerve paths, a person comes to associate the agreeing 
reports of the three analyses concerning pitch. As a 
result, one has the sensation of a pitch when the envelope 
of a higher-frequency complex wave has the periodicity 
associated with a sine wave that pitch. However, if the 
phase relations of the harmonies are altered, the sensation 
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of a low pitch may disappear because of the change in 
the shape of the envelope. Experiments have eliminated 
the possibility that the low pitch results from non- 
linearity and beats. Licklider’s theory appears to offer 
the first unified explanation of a number of auditory 
phenomena. Afterward, Schouten also discussed and 
demonstrated the phenomenon of the missing funda- 
mental, using three sine waves between 2 and 3 ke, sepa- 
rated by 200 cps and 200 cps to produce the sensation of 
200 eps. 

J.T. Allanson and I. C. Whitfield of the University of 
Birmingham then presented their paper on ‘“The Cochlear 
Nucleus and its Relation to Theories of Hearing.”’? On the 
basis of information concerning the inputs and outputs 
of the cochlear nucleus and available data about its 
structure, they attempted to determine the type of 
structure it must have in order to be able to transform 
the input nervous activity into the output. 

Thursday’s session ended with a talk by R. L. Gregory 
of the Cambridge University Department of Experimental 
Psychology on ‘‘An Experimental Treatment of Vision 
as an Information Source and Noisy Channel.’ He found 
an empirical extension of Weber’s law Al/I = C, where 
I is the level of illumination and A/ is the smallest dis- 
tinguishable increment, to include the effect of the areas 
A, and A, of the contrasting fields, namely, 


AI ; Alle 
Sim Cock Cit gece 


He regards the breakdown of Weber’s law at low inten- 
sities as due to “‘noise’’ and finds that the law may be 
extended to cover low intensities by rewriting it in the 
form Al/(UI + k) = C. The constant k les between 0.03 
and 0.04 foot-lamberts, but its relationship to the “noise” 
is not clear. Gregory also attempted to find a function 
describing the observed relation between area and thresh- 
old intensity on the basis of a threshold defined in terms 
of the probability of mistaking noise for a signal, etc. In 
the discussion period, Licklider pointed out that Tanner 
at the University of Michigan asserts there is no sensory 
threshold—merely a point below which the probability of 
detection is low. Licklider reported that if an image is 
stabilized on the retina, visual acuity is momentarily 
improved, but the image then disappears on account of 
fatigue. Small movements of the eye therefore do not 
appear to be responsible for the acuity. 


Fripay: BEHAVIOR AND Its MECHANISM 


J. T. Allanson of the University of the Birmingham 
University Electrical Engineering Department began the 
last day’s session with a talk on “Some Properties of a 
Randomly Connected Neural Network.” In order to 
permit a mathematical discussion of the behavior of such 
a network, he postulated a much simplified model, and 
derived expressions for the equilibrium rate of firing of 
neurons, the rate of damping of oscillations, ete., in a 
fixed randomly connected network, attempting to relate 
the results to various neural phenomena. Because many 
rather different networks exhibit very similar gross 
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behavior, Allanson concluded that electroencephalography 
is not likely to yield detailed information about the struc: 
ture of the central nervous system. 

Next, W. K. Taylor of the University College Depart: 
ment of Anatomy, London, talked on ‘“The Electrica 
Simulation of Some Nervous System Functional Activ- 
ities.”’ He described a number of networks built up ot 
model neurons and discussed the functional significance 
of their stimulus-response characteristics. Examples were 
included in which the correct response is “‘learned by 
association” rather than built into the network; a device 
was simulated which learns to turn right or left accordingly 
as rewards are found on the right or left. He concludec 
that the nervous system can be imitated by a network of 
interconnected units, most of which receive and deliver 
pulse-frequency-modulated signals at a multiplicity oi 
input and output terminals; other more specialized units 
handle the inputs and outputs of the network as a whole: 
Because the over-all input-output characteristics of the 
units are nonlinear and depend on past as well as present 
inputs, it is possible for the network to recognize selected 
features in visual or auditory patterns and to modify 
spontaneously generated sequences of activity in a man 
which simulates adaptation or learning. 

After the morning coffee interval, P. D. Wall deliverec 
a paper by himself, J. Y. Lettvin, W. 8. McCulloch, anc 
W. H. Pitts, all of M.I.T., on “Factors Limiting the 
Maximum Impulse-Transmitting Ability of an Afferent 
System of Nerve Fibers.” Experiments were conducted. 
1) to determine the maximum sustained frequency of 
nerve impulses which can be carried along a set of fibers 
originating in the skin, muscles, and leg joints, proceedigs 
into the spinal column, and streaming toward the head 
in the dorsal columns of the spinal cord; 2) to determine 
the limits on impulse transmission following a single 
impulse or short burst of impulses where all the impulses 
are carried by the same set of nerve fibers; and 3) tc 
determine the limits following activity in a neighboring 
parallel set of nerve fibers. The object of the investigatior 
is eventually to calculate the informational capacity of 
such a system and to ascertain how close to it the rate ol 
transmission of information in the nervous system comes. 
If it is close, information theory should provide insights fo1 
neurophysiology. The quantity of greatest interest is the 
mean frequency of impulses required to achieve the 
channel capacity, which is to be compared with the acta 
average frequency of impulses. 

After lunch, Dr. Oliver G. Selfridge of the Lincoln 
Laboratory, M.I.T., presented a very interesting papet 
“Pattern Recognition and Learning.’ The type of learning 
to which he referred is the evolution of a method fo1 
distinguishing patterns, which first requires a motivation 
and some sort of criterion (7.e., elementary method) for 
pattern recognition. In particular, he discussed the 
mechanical recognition of a quadrilateral as it might appea1 
on a snowy television screen. It may be distinguished from 
a triangle or circle by smoothing to get rid of spots, next 
eliminating everything but the boundaries between ligh: 
and dark areas, then eliminating everything but corners o: 
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he ends of edges, replacing each corner by a uniform blob, 
nd finally counting the blobs. Such experiments in data 
rocessing have been carried out on an automatic digital 
omputer. It is Selfridge’s feeling that, while a machine 
an be programmed to learn, there is little hope that a 
1achine will ever learn to play tolerable chess from the 
ules alone. In the discussion period, W. K. Taylor 
laimed a speed for his own analog equipment a million 
imes as great as that of Selfridge’s digital computer. 

Next D. E. Broadbent of the Applied Psychology 
vesearch Unit, Cambridge, England, under the title 
The Concept of Capacity and the Theory of Behavior,”’ 
resented a plea for the qualitative use of information 
heory in psychology, ‘‘to provide a language for talking 
bout events within the man’s nervous system.” In the 
iscussion period, the question of ‘“‘semantic hygiene” was 
aised in connection with this proposed use of exact terms 
1 new and vague ways. 

After the final tea interval, Henry Quastler of the 
Iniversity of Illinois Control Systems Laboratory spoke 
n “Studies of Human Channel Capacity.’’ He has found 
1 experiments with reading a single line of random piano 
usic involving an alphabet of 3 to 65 keys in which all 
otes are equally long, a maximum information rate of 
3 bits per second is achieved with a 25-key alphabet. 
No key is allowed to follow immediately after itself.) 
hysical limitations reduce the information rate when the 
Iphabet size exceeds about fifty keys or the required 
need exceeds about 5.2 keys per second. Quastler found 
hat the addition of more lines, accompaniment, words, 
r varied rhythm to the music did not permit a higher 
formation rate. This is to be contrasted with Licklider’s 
iscovery of a total rate of 35 bits per second for combined 
sading and pointing which separately can achieve rates 
f 25 and 15 bits per second, respectively. Quastler has 
ot been able to find two such compatible activities. The 
usks with which high rates are achieved, of course, are 
10se for which no thinking is required; the information 
ite in typing runs up to about 17 bits per second. 

Similar limitations seem to apply tasks of many types. 
uastler concludes that the maximum amount of informa- 
on of a single kind that can be handled at one time is 
5 to 5 bits, the maximum number of kinds of information 
vat can be handled together is about 7, and the maximum 
mount of information of all kinds that can be handled at 
glance is about 20 bits; the maximum information rate 
r sustained single activities is evidently about 25 bits 
ar second. These limits do not seem to be due to muscular 
nitations, since proofreading; e.g., which requires little 
uscular activity, goes at about 20 bits per second. Prac- 
ee can double the apparent information rate in piano 
aying though it requires twice the muscular activity. 
The last paper on the program was “Application of 
ommunication Theory to the Human Operator,” by 

D. North of Boulton Paul Aircraft Ltd., Wolver- 
impton, who discussed the applicability of game theory 
\d of information theory to the study of the character- 
‘ics of a human being in a control system. 

He was followed by a five-minute talk in English by one 
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of the Russians present, expressing his appreciation for 
the symposium and the bringing together of people from 
different countries and sciences; he also expressed his 
appreciation for Shannon’s work. It was his feeling that 
information theory will yield new high-speed means of 
communication. He reported that among those in the 
Soviet Union working in the field of information theory 
were: Kolmogorov; Khinchin; Yagvo (?); Patelnikov, who 
works on the theory of potential stability; Kharkevich, 
who is working on the accommodation of signals and the 
extension of Shannon’s work (he is reported to have pub- 
lished recently in Radiotechnika); Sardoniko, who works 
on automatic regulation; and the speaker himself, who 
works on the detection of weak pulsed signals and on 
active noisy networks. Finally, he expressed his apprecia- 
tion for the invitation to the symposium sent to his country 
and for the attention of the audience.’ 

Reprints of some additional papers were made available 
at the symposium and, though they were not on the 
program, they may be made part of the record. One of 
these is “A ‘Neuronic Model’ of the Inner Ear,” by 
Jérgen Koefoed of Universitetets Fysisk-Kemiske Institut, 
Copenhagen. Another was “Some Calculations on a 
Method for Signal Testing,’ by J. W. Cohen and M. M. 
Jung of Philips Telecommunication, Hilversum, the 
Netherlands. Their method is to regard a signal as present 
if the signal plus noise exceeds some criterion signal over 
a certain interval, and they have approximated the 
probability of error of this method. 

A count of papers presented at the symposium shows 
44 per cent came from Great Britain and 25 per cent from 
the U. 8. Most of the latter came from M.I.T., which 
appears to be the foremost center of work in the field 
of information theory. 


PUBLICATION OF PROCEEDINGS 


The proceedings of the 1950 symposium were published 
by the IRE, with the cooperation of the Ministry of 
Supply, in February, 1953. The Butterworths Scientific 
Press published the 1952 proceedings in September, 1953; 
and they are to publish the 1955 proceedings shortly. As 
before, the papers will appear in their formal form rather 
than as presented, and the remarks made during the 
discussions periods will be included. 

The large number of very stimulating papers presented 
during the symposium should serve to nourish the field 
of information theory until the next symposium is held, 
but perhaps the most important accomplishment of these 
symposia has been the bringing together of scientists 
from different fields who are using and extending informa- 
tion theory: physical scientists, philologists, psychologists, 
neurophysiologists, and mathematicians. The resulting 
cross-fertilization should prove to be a considerable 
stimulus to the further development of information 
theory and its applications. 


1 ®. C. Cherry, “Information Theory: Third London Symposium,” 
Nature, p. 773; October 22, 1955, identifies the speaker as V. Siforov. 
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Some Optimal Signals for Time Measurement 


HERBERT SHERMAN T 


Summary—Some optimal transmitted signals are given for com- 
munications systems in which the time of occurrence of a signal is 
desired in the presence of additive Gaussian noise. These signals 
are of two classes; those in which bipolar signal excursions are 
permitted and those in which only unipolar signal excursions are 
permissible. 

Some bipolar codes used by R. H. Barker are supplemented when 
conditions permit unlimited negative excursions of the optimally 
correlated signal output. Other constraints, such as limiting the 
positive and negative excursions of the optimally correlated signal 
output, except at the proper phase, will lead to different codes. 

When only unipolar codes are permitted, optimum repetitive and 
nonrepetitive codes (embedded in zeros) are given for various code 
lengths. The minimum length of such codes is given. A mathe- 
matical resemblance is indicated to a frequency allocation problem 
in which third-order intermodulation products must be avoided. 

The closely related concepts of error-detecting and correcting 
codes and optimally correlated signals are illustrated in the deriva- 
tion of these codes. The problem of generating such codes by other 
than trial and error is not solved and remains as a provocative 
problem. 


INTRODUCTION 


N CERTAIN communication systems, the time of 
occurrence or phase of the signal is the parameter 
which is sought in the presence of additive Gaussian 

noise. Some examples of such systems are radar ranging, 
certain navigational systems, PPM systems for voice 
communications and radar azimuth determination. The 
ideal signal for such systems are signals which are ortho- 
gonal for each possible time position. Such orthogonal 
signals do not overlap in time. If the received signals are 
permitted to occupy all possible time positions, the optimal 
transmitted signal is an impulse function. If the received 
signal is permitted to occupy only quantized time positions, 
then the optimal transmitted signal is time limited to the 
permitted time positions. 


SomE OptimAL SIGNALS FoR TimE MEASUREMENT 


In many practical cases orthogonal signals are not 
always possible due to physical constraints. In the radar 
ranging case, for example, peak power limitations may 
preclude the short pulse transmission consistent with 
desired accuracy and available average power. Is there a 
signal configuration which will make best use of the 
available energy and also give the best possible phase 
determination? 

In linear systems it has been established by previous 
writers’ that the pre-detection cross-correlating filter or 
“matched”’ filter (suitably modified for Gaussian colored 


* The major portion of this paper is Chapter VI of a thesis titled 
“The Detection and Phase Determination of Signals in Additive and 
Multiplicative Noise,” submitted in partial fulfillment of the re- 
quirements for the degree of Doctor of Electrical Engineering at 
the Polytechnic Institute of Brooklyn. 

¢ Lincoln Lab., Mass. Inst. Tech., Lexington, Mass. 

1 See, for example, B. Dwork, “Detection of a pulse superimposed 
on fluctuation noise,’ Proc IRE, vol. 38, p. 771; July, 1950. 


noise) takes advantage of all of the available energy to 
create the best peak signal to rms noise ratio in the pre- 
sence of Gaussian additive noise. Thus, independently of 
how the signal is shaped in time (providing it is of finite 
length), we can recover all of the energy into a signal 
peak, subject only to the difficulties of physical construc- 
tion. Such a filter might take several forms. One form 
would be a network whose impulsive response is the 
reverse in time of the desired signal. In the presence of 
white noise, the filtered output peak signal squared to 
average noise power ratio will be 2H /W, where F is the 
total signal energy and W, is the ‘‘white’’ noise pow 
per cps at the filter input. 

The peak signal from the cross-correlating filter will 
indicate (except for time delay through the filter) the 
time phase of this signal in the absence of noise. Noise 
will tend to introduce false peaks or to detract from the 
signal peak. It should therefore be the object of an opti- 
mum signal-filter combination to maximize the amplitude 
separation between the output signal peak and the ‘‘side 
lobes” or leading and trailing edges of the filtered signal 
output in the absence of noise. 

Consider, for example, a pulse-modulated carrier wave 
which is sampled at twice the carrier frequency and 


denoted in quantized form | 


0.0) - 1 =—T i el OO ! 


A typical output of the optimum filter for this signal 
would be 


-00 —1 +2 —3 +4 —5 +6 —5 +4 —3 +2 —100 =e 


The addition of two units of noise at the (+4) point 
would create an ambiguity with the (+6) peak signal, 
while more noise would give a false indication of phase by 
one or more cycles of the carrier frequency. It is possible 
to find better signal shapes than this, but some preliminary 
remarks are in order first. 

All of the codes proposed presume that the signal can. 
be given a “fine” structure. In the case of a radar pulse 
this fine structure may be, for example, 1) frequency or 
phase modulation within the pulse envelope, or, 2) the 
division of a long pulse (to increase the average power) 
into a number of shorter pulses having the same total 
energy. 

It is assumed that a peak power limitation exists and 
that this power is used fully. There appears to be no 
remarkable advantage in varying the signal intensity 
below the peak power level. 

The length of the signal may be set by several considera- 
tions. The minimum length of the signal may be set by 
the time necessary to transmit the available energy under 
constraints of a specified peak power. The maximum 
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length of signal may be set by problems of resolution 
between overlapping signals or by the time needed to give 
the signal the desired character necessary to optimize the 
eross-correlated output or by the maximum repetition 
rate if the signal is quasi-periodic.” The time necessary 
to give the signal the desired characteristics is discussed 
in the generation of the optimum codes. 

Two general types of codes are considered. One may 
be used where bipolar signals are possible,* as in radar 
systems prior to envelope detection. The second may be 
used where only unipolar signals are available as is the 
case after envelope detection or where the signal may be 
generated serially from pulsed phase-incoherent sources. 
Ideal bipolar codes have been considered by R. H. Barker.* 
Unipolar codes have arisen in certain frequency assignment 
problems and have been considered by Babcock* and 
Reiffen.” 

Both codes attempt to place the bulk of the signal 
energy in as wide and flat a power spectrum as possible 
by narrowing the width of the varying portion of the 
signal autocorrelation function. 


BreoLtarR Copes ror PHAse HRROR CORRECTION 


In the absence of noise, the output ¢(7) of an ideal 
white noise resistant cross-correlating filter whose desired 
input is S(t) will be 


(7) = ap! S()S(t — 2) dt. 


In the absence of signal and noise, the output of the filter 
will naturally be zero. It is apparent then that the maxi- 
mum amplitude separation we can usefully achieve 
between signal and no-signal or between signal and 
phase-shifted replica, is that between the output signal 
extreme and zero signal level. We would like this to be 
achieved at some interval (which will also condition the 
occupied frequency spectrum) consistent with our desired 
smallest interval of phase determination. This can be 
expressed as an integral inequality. 


[ S(OS(E — 7) dt < 0; [Papel mars 


The negative excursions of the auto-correlation function 
[#()] will be important where multiple signals are present 
since they may combine with the noise to subtract from a 


2An assumption is made here that Doppler frequency shifts 
generated by motion of the transmitter and receiver are small 
(e.g., the period of the Doppler frequency shift is long compared to 
the signal length). Woodward and Davies discuss uncertainty 
relations due to Doppler shifts. In addition, while this paper was in 
manuscript form, unpublished work by Siebert and others at Lincoln 
Laboratory, M.1.T., concerning the uncertainty effects of Doppler 
shift and on ternary codes similar to those described hereafter came 
to the attention of the author. 

3R. H. Barker, “Group synchronization of binary digital sy- 
stems,” “(Communication Theory,” W. Jackson, Ed., Academic 
Press, New York, N. Y., p. 273; 1953. 

4W.C. Babcock, “Intermodulation interference in radio systems,’ 
Bell Sys. Tech. Jour., p. 63; January, 1953. 

°>B. Reiffen, “A System for Minimizing Intermodulation Inter- 
ference in Communication Systems Based on Frequency Selection,” 
Lincoln Lab. Group Rep. 311-5, July, 1955. 
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signal pulse and cause detection failures. For this purpose 
it would be desirable but not always possible to require 
that the output signal be identically zero for all | + | > 7. 
This may be inconsistent with the time required to trans- 
mit the desired amount of energy with limited peak power. 
Some negative excursion will therefore be found necessary. 

The integral inequality can be reduced to the following 
form for substantially band-limited (W) signals by 
sampling S(t) at S(t) = S,, S(t.) = S., --- , where 
S(t) = 0;¢ > tyjt. — t, = 1/2W and considering values of 
t which are integral multiples of the sampling period 


Sye1 ~ 0 
Sy is Sy-iS; Ss 0 
SwS3 ta Sy-1S2 BE Sw—281 <= 0 


SySw-1 ae Sw-1Sy-2 ae chee ar SS, — 0. 


There are N unknowns to be found from N — 1 equa- 
tions with N — 1 adjustable constants (on the right-hand 
side). It is helpful and physically acceptable to restrict 
S,; to values of +1, —1, and zero (a ternary code). 

Solutions exist for all values of N > 2. This existence 
theorem is readily demonstrated by allowing 


Sy == 
Sy = 


=k 
eral: 


These values separated by a string of zeros will always 
satisfy the inequalities given above. This is a trivial 
solution however. At least two extremes are interesting. 
In the first it would be desirable to have every pulse 
position occupied (by other than zero). This will permit 
the maximum peak in the auto-correlation function in the 
shortest total signal time interval. R. H. Barker’ has 
sought these solutions and found them for N = 3, 7, and 11. 

Failing this, one would like to maximize the distance 
between output signal peaks and “‘side lobes”’ of the corre- 
lated signal output. This implies that one should consider 
“side lobes”’ in the correlated signal output other than zero 
and, in fact, should make the maximum off-peak signal 
output a dependent variable. Limitations of manual 
computation precluded exploration of some of these 
byways. 

Another extreme among the possible solutions is to 
require 


SwS1 Ss —| 
SwSo ae Sw-181 = 0 


SwSy-1 aie Sy-1Sy-2 SF as ata SoS; = 0. 
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TABLE I 
“Oprimum’’ TERNARY CODES 


Nos es 
*N =3 Sats —- 0 + 

Nestle eo 20a ae 

Ne=5 - 0 -— + + —- 0 + 0 + 

NE—nG - + - 0 + + —- 0 0 0 0 + 
INS SS POSE = © OW = © © 4 

hy = 8 = eS 0 OO = = 

N=9 = +)5-"O -0°% OF +540 =] 0640900 =) 07 = sO 0 OR Oe Oe On Oe 
* Nice ee 5 ee ae ee gl 

+, — are pulses of unit amplitude and given sign. Mirror images are not shown. 
* The codes having no zeros are Barker’s. 
TABLE II 
AUTOCORRELATION FUNCTIONS FOR OpTIMUM TERNARY CODES 

N= 2 = S74 

Ne=s3 — 0 +8 —- QO +2 

N =4 = O @ se 

N= 5 - -—- 0 0 +4 - 0 0 0 +3 

= —-— 0 0 0 = +5 = W WM OO 0 < 

N=7 — 0 — 0 = O =7 => ) © WM OQ @O = 

N=8 — 0) 105 — 250 05 0 6 

N=9 =<0° 0-70 = 90 OF = +6 —=90 = 0 0) 0) OF 0S 9 500 OM Oe Oe Oe Ome ces 
Nie = 0 = 0 =| 30) 0) =~ 207 Fil 


This approximates the condition one might prefer where 
the ‘‘apparent” resolution of super-imposed signals is a 
problem. 

The solution of the array of inequalities is somewhat 
of a chore. The number of possible solutions is 3”. For 
N = 9, this implies 19,683 possibilities. Trial and error 
should be avoided if possible. 

While some consistent procedures for solving these 
inequalities were evolved, these involved a good deal of 
trial and error and led to solutions for optimum ternary 
codes of the former type, given in Table I along with 
Barker’s codes. The autocorrelation functions for these 
codes are shown in Table IJ. The auto-correlation func- 
tions are symmetric about the maximum. Only half of 
the autocorrelation function is therefore shown. 


Binary HRROR-CORRECTING PHASE-DETECTING CODES 


As an approach to the determination of an optimum 
signal for phase-detection where only unipolar signals 
are available, the concept of a binary error-detecting and 
correcting code similar in function to the previously 
discussed ternary code will be used. In the case of certain 
multiplicative noise, these unipolar binary codes become 
most appropriate for phase discrimination.° 

Ideally it would be desirable to find a binary code 
sequence which would, in the presence of digit errors, 
unerringly indicate the correct phase of the signal. Un- 
fortunately one can only approximate such a sequence 
by selecting the sequence which will correct for one or 


6 H. Sherman, loc. cit. 


more errors. We therefore seek a binary code sequence 


which will unambiguously indicate the phase of a signal | 
in the presence of errors of less than a given maximum 


number. 
The phases of a cyclic signal sequence are given by one 


of all possible sequences generated by advancing each | 
signal simultaneously by one pulse position and feeding | 


the “carry” digit into the start of a sequence. As an | 


example, all possible phases ¢ of one sequence are: 


pi Del s07070 
gs OR Teor. 
d3 OMOET a En. 
ps OO70F ISI 
ds LOPOs0 St 


It will be assumed hereafter that each signal may occupy 
all possible phases. The number of phases which can occur 
is the sequence length. A noncyclic group of digits is hereafter 
called a message. Its phase is advanced by one digit 
movement of the entire message. 

For the purpose of accounting for error we shall use the 
concept of “distance” between codes. The distance be- 
tween two binary sequences is equal to the minimum 
number of digit changes required to convert one sequence 
into the other. The concept of a minimum distance as a 
measure of correct signal choice is identical to choice of the 
maximum cross-correlated signal in the case of phase 
discrimination. The use of distance in this section happens 
to be convenient for the derivation of optimum binary 
codes. The relationship between the usual definition of 
squared distance (hereafter called ‘distance’’) 


: 


d°(7) Ds DS) eS (tees) |, 


and the number of digit changes can be demonstrated for 
binary codes by making the physical subtraction required 
by [S;(7,) — S;(72)]. When the digits are identical this 
term goes to zero; when the digits are unlike, the differ- 
ences are +1, which when squared and added leads to 
the definition given above. 

If we expand the bracket and combine terms 


d = 2[)) 8; — D7 SSE + 9], 


we can recognize ‘distance’ as twice the difference 
between the peak of the autocorrelation function and the 
value of the autocorrelation function at 7. 

The maximum number of errors which can be tolerated 
is less than half of the minimum ‘‘distance’’ between 
phases. If the number of errors exceed this, the received 
message may be ambigious (exactly half-way between 
two possible messages) or erroneous (closer to the wrong 
message). 

One possible statement of the problem is to fix the 
number of 1 (signal) digits (similar in certain cases to 
fixing the signal energy), which are here assumed to be 
less than the zero digits, and to determine the sequence 
length and digit sequence in which the maximum number 
of digit errors can be corrected. For this message the 
“distance” between all possible phases of the message 
shall be equal to or greater than a given number. 

The problem is nontrivial only in the case where 
successive signal phases overlap (7.e., the phase difference 
between signals is less than the message length where 
the message length refers to the smallest number of digits 
which encompasses all 1 digits in any phase). When the 
signal phases do not overlap, then the “distance” between 
signal phases is always equal to twice the number of 1 
digits in the message and is independent of the disposition 
in | digits and the length of the message. 

Thus ¢, and ¢, are two signal phases (nonoverlapping) 
which are always four units apart in either of the following 
cases 


od: Lete0,.0.07 0) 

do OFOLO 1213.0 
or 

>, heOnsis. 05050 

2 OF ORO RIS Ooi 


For overlapping, phase-shifted binary sequences, the 
“distance” between least distant phases can be no greater 
than twice the number of 1 digits (assuming these are less in 
number than zero digits) less 2. 

The greatest ‘distance’ between two identical (except 
in phase) binary sequences is twice the number of 1 digits, 
if the 1 digits are fewer in number than 0 digits. The 
signal, while phase shifted through all possible phases, 
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must have at least one position in which a digit coincidence 
occurs between phases. In general the distance between 
two. phases is twice the number of 1 digits less twice the 
number of 1 coincidences between phases. If a minimum 
of one digit coincidence occurs, then the distance between 
closest phases is as stated above. 

Let us examine an example of an error-corrected code. 


Example Number of signal digits = 3 
Sequence length = 7 
Maximum digit error tolerated = 1 


Minimum distance between phases = 4 


I ©. © OC 
OCS Oat Ong 
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OO cOFIeaOrt 
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The generation of such codes may be accomplished by a 


trial and error procedure.’ 

It will be useful to characterize the successful code by a 
series of decimal digits which denote the digit positions 
from a | digit to the next succeeding | digit. If the sequence 
were cyclic as in 


L160) 1205008 OZIZOL0 x0 Sims 
then the decimal sequence would be 
1-2-4-1-2-4-1-2 


This particular sequence will therefore be denoted as a 
1-2-4 sequence (or 4-1-2 or 2-4-1 or inverting, 4-2-1, 1-4-2, 
2-1-4). [t 1s emportant to note that this sequence, as well as 
all succeeding optimum codes, 


1) has no repetition of decimal digits within one cycle, 

2) is such that the sum of any two or more contiguous 
decimal digits is unequal to any other digit in the 
sequence or to the sum of any other two or more 
contiguous decimal digits. 


This condition is necessary since its violation would 
permit two digit positions tc coincide in two phases 
reducing the minimum distance by an additional 2 digits 
from the optimum. 

The example given above was for an optimum error- 
correcting code of minimum length for a cyclic repetition 
of the sequence. As a result of the requirement that the 
sum of contiguous decimal digits (characterizing the 
spacing of | digits) shall be unequal to any other sum of 


7 The reviewer has pointed out a paper presented to the 1955 
Third London Symposium on Information Theory by D. A. Huffman, 
“The Synthesis of Linear Sequential Coding Networks,” which 
presents methods for finding shorter error-correcting codes (known 
therein as “maximum length null sequences’) in which the ones 
and zeros are nearly equal. Such codes are useful in the important 
cases where signal energy is dependent only on the length of the 
code rather than on the number of one digits. 


28 IRE TRANSACTIONS—INFORMATION THEORY March‘ 


TABLE III 
Brnary Error-CorrRECTING CopDES 


Max. Min. Cyclic Min. 


Hrrors Min. No. of 1 Sequence Decimal Binary Code Message Decimal Binary Code 
Corrected Distance digits Length Code Length Code 
0 2 1 2 0 01 1 0 1 
0 2 2 3 1-2 Wako 2 if 1] 
1 4 3 i 1-2-4 1101000 4 1-2 TOME 
2 6 + 13 1-3-2- 1100101000000 a 1-3-2 II OLOSSOR 
3 8 5 21 1-3-10-2-55 110010000000001010000 12 1-3-5-2 110010000101 
a 10 6 31 1-3-2-7-8-10 18 1-3-6-2-5 


contiguous decimal digits we can establish the minimum has found the most dense frequency allocations (error- 
length of such codes. Since each sum must be different correcting codes in this context) for up to 10 frequencies 
from all others, the number of sums which can be formed (digits). These are reproduced in Table IV. Mr. B. 
from an N digit sequence length is 
TABLE IV 
1) the decimal digits taken singly = N Most Compact Copzs up To 10 Drarrs* 
2) the decimal digits taken in contiguous pairs = N EMBEDDED IN A SEQUENCE OF ZEROS 


3) the decimal digits taken in triplets = N 


The number indicates the location of 1 digit. 


ae 
Only taG 
. Oreo Ai 
4) the decimal digits taken in ““(N — 1) lets” = N y ? a i s on 
5) the decimal digits taken N at a time = 1 0, 14 ORS oo hse 
0, 1, 4, 13, 24, 30, 38, 40, 45 
The minimum number of sums is therefore 0, 1, 7, 11, 26, 39, 47, 96, 59, 64 
L = NWN -— 1) +1. * W. E. Babcock, loc. cit. 


There can be no fewer sums than this or two would be 
equal. Since the largest sum of contiguous decimal digits 
is equal to the sum of all decimal digits (the length of the 
sequence), this largest sum must equal or exceed L in 


order to have room for the minimum number of sums, L. AL etm ne Eee tn Cupolas: 


The length of the sequence must therefore be greater 0 30 147 400 821 ©1416 
than or equal to L. lc) AL (ASL GAA 22 
If the sequence can be embedded in a number of zeros 3 65 203 564 969 1650 
such that the sequence length exceeds the minimum above, 4 80) 9» 25125927 1015 
then we can group our 1 digits to have shorter message 12 96 = 289.1661) Wilkos 
lengths. Some examples of both are given in Table III. 2091225 SOOT Bis 


The mathematical problems of finding the optimum 
binary error-correcting code for phase determination, 
where the code is embedded in zeros, is identical to that I am indebted to Professor A. E. Laemmel of the 
of specifying the allocation of frequencies which are free Polytechnic Institute of Brooklyn for the suggestion of 
of third order intermodulation products. This problem the error-detecting code as an approach to the optimal 
has been considered by two authors. Mr. W. Babcock* signal for phase determination. 
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Reiffen” has found sets (not the most compact sets) of — 
frequency allocations for up to 33 frequencies (digits). | 
The signal digits in Mr. Reiffen’s set are located in the | 
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An Analysis of Signal Detection and 
Location by Digital Methods’ 


G. P. DINNEEN}+ AND I. S. REEDt+ 


Summary—An analysis of the detection and location of repetitive 
gnals in noise by digital techniques is made. The problem of loca- 
on of the center of signals, herein denoted as beam-splitting, is 
xplored. A Monte Carlo method employing a high speed digital 
ymputer was used to obtain quantitative results for a variety of 
igital detectors. A method of mathematical analysis is described 
nd used to check computed results. The work described differs 
om much of the previous literature on detection or statistical de- 
sion theory in that an estimate of signal location is demanded. 


INTRODUCTION 


ECENTLY the detection of repeated signals in 
noise has been aided by signal-integration schemes 
using digital methods for signal storage.’’” These 

schemes have generally employed a method of integration 
hereby quantized signals are added. Although it is 
ossible to quantize the signal amplitude into many dis- 
rete levels, we shall consider quantization into two 
mplitude levels and between fixed time markers. If the 
riginal signal plus noise waveform exceeds a_prede- 
sarmined amplitude between given time markers, a stan- 
ard pulse is generated at the end of the interval; if the 
awreshold is not exceeded, no pulse is generated. The 
robability of obtaining a standard pulse can then be 
etermined and the probability of the sum of pulses over 
fixed interval is given by the binomial distribution. The 
dvantage of these digital methods over a corresponding 
nalog scheme is the possibility of remembering signals 
ver a large number of samples. Thus the increase in 
enal-to-noise ratio as the square root of the number of 
imples can be fully realized. As the number of samples is 
creased however, one suffers a loss in accuracy, since we 
ow know only that a signal is present somewhere within 
1e sample interval. Moreover, if the signal straddles two 
tervals we have an added ambiguity. In order to over- 
me these disadvantages, a scheme for locating the 
nter of the signal in addition to its detection is called for. 
For purposes of exposition, the quantizer can be con- 
dered to deliver a one if the signal is above the quantizer 
reshold, and a zero if below. Thus the quantizer may 
>» regarded as transforming a radar sweep into a sequence 
“ones and zeros, where each digit is delivered to the 


* The research in this document was supported jointly by the 
my, Navy, and Air Force under contract with the Massachusetts 
stitute of Technology. 

+ Mass. Inst. Tech., Lincoln Laboratory, Lexington, Mass. 
1J. V. Harrington and T. F. Rogers, “‘Signal-to-noise improvement 
rough integration in a storage tube,’”’ Proc. IRE, vol. 38, pp. 
97-1203; October, 1950. 
2J. V. Harrington, “An analysis of the detection of repeated 
mals in noise by binary integration,’ Trans. IRE, vol. IT-1, 
. 1-9; March, 1955. 


detector at the quantizer time intervals. Imagine now 
that each quantized sweep sequence is laid parallel in 
time sequence from left to right in such a manner that the 
information at a given range is along a horizontal line. 
Time measured in sweep number would be the abscissa, 
and range would be the ordinate of this picture (Fig. 1). 
A target would appear as an increased density of ones at 
a given range throughout an interval of sweeps corre- 
sponding to the beamwidth of the radar. 


Fig. 1—Quantized radar video. 


If a machine has enough memory to hold the target 
information for several sweep times, on the order of a 
beamwidth, then an estimate of the density of ones can 
be made. If this estimate of density exceeds some threshold, 
one can say that a target has been detected. The number 
of false targets detected by this means can be kept low 
by a proper setting of the quantizer threshold. 

If changes in density, as well as the density, can be 
discovered, an estimate of the azimuthal positions of the 
leading and trailing edges of a target sequence at some 
range can be obtained. By this information, it is clear 
that an azimuthal estimate of the center of the target is 
obtained. In fact, if A@ is the azimuth increment between 
leading and trailing edges of the target, and if @ is the 
azimuth of the trailing edge, then 6 — A@/2 is an estimate 
of the target center. The general process discussed above 
has been termed the beam-splitting process. 

It is the purpose of this paper to discuss various methods 
for detection and location of signals in noise. This repre- 
sents a natural extension of the digital integration scheme 
discussed in Harrington’s paper.” 


SIGNAL-TO-NoIsE Ratio 


For purposes of exposition, let us consider the threshold 
detection of a sine wave in additive Gaussian noise. The 
probability density for the envelope Rh of a sine wave 
plus noise (P sine wt + I,), where the noise is confined to 
a relatively narrow band centered on the sine-wave 
frequency, 1s 


DR = Rd k 


Wo 


exp [—(R? + Py/2vour#P), (1) 
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where WY is the noise power and J, is a modified Bessel 
function of the first kind and zero order.® For quantization 
in time of the order of a pulsewidth, the probability of 
obtaining a quantized video pulse, given a pulse of ampli- 
tude P in the IF amplifier, is 


co 


Ppa i} Vd V exp [—(V? + 0)/21o(aV), 


. 
where the variables have been normalized with respect 
to the noise power; i.e, V = R/V, the predetector 
signal-to-noise ratio a = R/V, and V is the normalized 
amplitude threshold in the quantizer. 

In the special case where a signal is not present (a = 0), 
the probability of obtaining a quantized video pulse due 
to noise alone is 


P, = PO, V) 


I 


iP Pale 7 2 ea ae ee) 


Using these two equations, the functional relationship 
among a, P,, and P, may be deduced. This is shown 
graphically in Fig. 2. 
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Fig. 2—P, = probability of obtaining a quantized video pulse due 
to signal, P, = probability of obtaining a quantized video pulse 
due to noise alone. a = predetector signal-to-noise ratio. 


Hence, in later discussions, when reference is made to 
predetector signal-to-noise ratio, the a used in the equation 
above is implied. This model has been used to provide 
some measure of signal strength for various settings of 
P ands. 


TarGET DESCRIPTION 


Let us first remark that the quantization into zero or 
one, of course, destroys all amplitude information except 
whether or not the signal exceeds a certain threshold. 


38. O. Rice, “Mathematical analysis of random noise,” Bell 
Syst. Tech. Jour., vol. 23, pp. 282-332; July, 1944; vol. 24, pp. 
46-156; January, 1945. 


Mare. 


Therefore, precise information about the beam patter; 
is not useful unless a real symmetry is apparent; that is 
unless the density of ones over some interval less than th, 
beamwidth increases as one approaches the center of th, 
target and decreases on the far side. In many cases, th. 
fading and scintillation causes enough gaps in the targe 
so that this symmetry does not occur. Therefore, it 1 
assumed that the primary indication of the presence of , 
target is a sharp increase in the real density of ones u 
some interval corresponding to the beamwidth of th 
radar antenna. 


MATHEMATICAL MopEL 

A mathematical model of the target and noise region| 
after quantization can be given in terms of Bernoull 
sequences. A Bernoulli sequence is a sequence of trials 
each of which has two possible outcomes with proba 
bilities p and gq = 1 — p. For convenience, the outcome; 
will be denoted by one with probability p and zero witk 
probability gq = 1 — p. A target region is then a subse, 
quence of a Bernoulli sequence with p = P,, and a nois 
region is then a subsequence of a Bernoulli sequene 
with p = P,,. 

Consider one of the quantized sweep sequences cor 
responding to a given range as in Fig. 1. Azimutha 
positions will be measured now in sweep numbers. Thy 
target that appears as an increased density of ones ha, 
a width in sweep numbers corresponding to the beamwidtl 
of the radar antenna. The mathematical model for one o' 
these fixed range sweep sequences is a pair of Bernoull 
sequences. The quantized radar video, that is, the zero-on 
sequence, corresponding to the noise region, is representec 
by a Bernoulli sequence with p = P,, and the zero-on 
sequence, corresponding to the target region, is repre 
sented by a Bernoulli sequence with p = P,. 

This description of the target region emphasizes it 
statistical nature. For example, if the average number o 
hits per beamwidth is 16 and p = P, = 0.31, then Fig. ¢ 
is a plot of the probability distribution of real densities 
This is, of course, Just the binomial distribution. Crosse¢ 
points are those obtained by means of the stochasti 
process carried out on Whirlwind I, described later. 


Bure-Scan Ratio 


The blip-scan ratio is the ratio of visible returns fron 
a target to the number of scans where blip is used t 
denote visible return. For a human observer and a scope 
this means the number of times the observer sees th 
target divided by the number of scans or the number o 
times when observation is possible. The integration ove 
a beamwidth is then performed by the scope and by th 
eye of the observer. Experimental data are usually taken 
for a particular radar to plot the dependence of the blip 
scan ratio against signal strength or signal-to-noise ratic 
This curve will be a function of the radar and of th 
observer. 
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For this study, the integration over a beamwidth is 
erformed by the detector. The final blip-scan ratio is 
vain dependent on the radar receiver and also on the 
haracteristics of the detector. The value of P, which we 
1all use, defined as the probability of a one due to signal 
lone, may be analogously called the average hit-sweep 
atio. P, may be considered as the mean of the ratio of 
be number of ones returned from a target to the total 
umber of sweeps per beamwidth. Fig. 3 shows the distri- 
ution of the hit-sweep ratio for an assumed value of 
», = 0.31; that is, out of 1,000 targets, about 210 will 
ave a hit-sweep ratio of 5/16. The blip-scan ratio for a 
etector will be the probability of detection, or Q, as it 
rill be denoted later. The curves that will be drawn later 
ill give the blip-scan ratio as a function of P,, that is, 
he average hit-sweep ratio. It is interesting to note that 
‘one plots the blip-scan ratio vs the actual (not average) 
it-sweep ratio, the graph has sharp discrimination and 
mphasizes the essential nature of all the detectors 
tudied, which is that of measuring real density over some 
rbitrary interval. 
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Fig. 3 


THe DETECTORS 


Since a target region is distinguished from a noise 
egion primarily by its density, the design of a digital 
letector for quantized video is equivalent to the problem 
f measuring the density over some interval determined 
yy the beamwidth. The detector must also be physically 
ealizable, which means primarily that its storage require- 
nents are not excessive. 

This problem is much more difficult than would appear 
rom the above formulation. The detector, of course, will 
ave no a priori knowledge of the target beginning. 
‘herefore, it must be sufficiently sensitive that it detects 
he increased density region quickly, but not so sensitive 
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that it detects false targets due to noise. If it does detect 
a target beginning, the detector must perceive the end 
of the increased density region. However, if it is too 
sensitive to this change, it will have a tendency to split 
targets. The detector design criteria can be stated as 
follows. 


1) It must detect the maximum number of targets with 
the minimum number of false alarms. 


2) It must have quick decision time on target beginning, 
but must not exceed a prescribed number of false decisions. 


3) It must have quick release time in noise, but must 
not exceed a prescribed number of split target decisions. 


It is clear that the design of a detector must be a com- 
promise because of the conflict in the design criteria and 
the constraint of small memory. 

Although it is possible to make a theoretical analysis 
of many special detectors, much remains to be done along 
these lines. In particular, it would be desirable to have 
empirical relations between such quantities as beam- 
splitting accuracy and detection sensitivity, beam-splitting 
accuracy and storage capacity, etc., which would apply 
to large classes of detectors. 

For this paper, three classes of detectors have been 
chosen on the basis of previous theoretical and experi- 
mental work. These detectors and their modifications will 
be described in the following sections. 


Success-Run Detector 


It is possible to calculate the probability of occurrence 
of a specific run; 7.e., configurations of zeros and ones, in 
a Bernoulli sequence of zeros and ones.* The mean recur- 
rence time, or the time between happenings of these 
runs, may also be calculated. Certain runs have a high 
probability of occurrence, hence a short mean recurrence 
time in a noise region; and a low probability of occurrence, 
hence a long mean recurrence time in a signal region. Thus, 
a simple success-run observer would count the recurrence 
time of some event and make a decision between target 
and noise based on the magnitude of this count. This 
would be done most simply by setting a threshold and, 
if the count exceeds the threshold, the observer detects a 
target, otherwise noise. In contrast with Feller’s studies,* 
overlapping runs will be considered. Thus, if the event is 
0 0 0, it is said to occur 4 times in the following sequence 


000001000 
pe ha t 


For example, consider the following, where S measures 
basic time units (sweeps in the radar case, V is the result 
of the Bernoulli trial (quantized video in the radar case) 
and L is the counter which measures recurrence times. 


4W. Feller, “An Introduction to Probability Theory and Its 
Applications,” John Wiley and Sons, Inc., New York, N. Y., 1950. 
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In this case the L-counter, or recurrence-time counter, 
reaches 16 before it is reset to zero by the occurrence of 
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the event 0 0 0. On the other hand, if the Bernoulli, 


sequence has a small probability p = P,, 
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The largest count is 6. Thus, experimentally one might then 
set a threshold at 7 so that, if the counter does not reach 7, ee st ea 
the decision is noise, but if it exceeds 7, the decision is sae : ie ‘A ; 
target. To find the center of the Bernoulli sequence one A target is detected when J, exceeds 6. The target 
takes L/2 and subtracts from S. beginning occurs for l, = 1 and the end occurs when 


The modified success-run observers detect on the basis 
of the recurrence of one of several events. The assumption 
is made that the observer is trying to detect a target in 
noise so that the L-counter is set at zero and remains at 
zero until a one is received. It then looks for the occurrence 
of one or more events /, throughout an interval L,. If 
Ei, occurs, the counter is reset to zero and the process 
begins again. If #, does not occur, the detector searches 
for an event or events H#, through an interval L,. If E, 
occurs, the counter is reset to zero and the process begins 
again. If #, does not occur, the detector searches for an 
event or events #7, through an interval L3, etc. Based on 
an a priort probability of the length of the target region, a 
threshold T is set. Whenever the counter is reset to zero, 
the value 1, of the L-counter just prior to resetting is 
compared with 7’. If J, < 7, it is assumed that the count 
is due to noise and no action is taken. If 1, > T, it is 
assumed that the count is due to signal, and 1,/2 is sub- 
tracted from the value of the S-counter, or sweep counter, 
to give the location of the center of the target sequence. 
Different detectors are obtained by choice of the intervals 
L,, Lz, L3, «+ , the events Hy, H,, --- , and the threshold T. 

A special case will now be described, but first the opera- 
tion of one of these modified detectors will be contrasted 
with a simple success-run detector (Fig. 4). 

Success Run A. The L-counter of Success Run A is 
initially set at zero. When a one is received, the L-counter 
is set to one at the next unit (sweep). The L-counter 
obeys the following rules. At each unit time (sweep), an 
observation Of pna—sHn—2én—-1M, and I, is made, where uy, 
is the value of the Bernoulli trial at the nth sweep, and 
lL, is the value of the L-counter at the nth sweep. If 


1), 0.5.) <eopeand either cians he Oe ORO 
or Bn —2Hn—1bn = 0 000 

PAL Bee | Sag and Para ihe =10"000 
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lL, > 6 and l,,, = 0. There is, therefore, a natural bias of 2.5. 
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Fig. 4—QV = Quantized video in given range interval. Success-run 
detector reset when 4 successive zeros occur in QV. Modified 
success-run detector reset when: 1) Last 3 QV pulses are 000 or 
100 and/ < 3, 2) Last 3 QV pulses are 000 and 4 < e < 7, 3) Last 
4 QV pulses are 0000 and 7 < 1 < 63. 


Success Run B. At each unit time (sweep), an observa- 
tion Of ,—4ln—3ln—sle<ity BDO IS manent 


1) 0 s bi < 4 and either Mn—-3hn—2Mbn—1én = 0 1 0 0 
or Mn—2bn—1én =000 
DANE ee) tay and fis 55] Voy sn] Us = 000 
3) 7 < (ps and Mn—4bn—3bhn—2bn-1bhn = 0 0 0 0 
then 
Li+:1 = 0; otherwise J,., = 0 + 1. 
A target is detected when l, exceeds 7. The target 
beginning occurs for J, = 1, and the end occurs when 


lL, > 7and l,,, =0. There is, therefore, a natural bias of 2.5. 
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equential-Observer Detectors 


The sequential-observer detectors are a class of detectors 
at consist of an up-down counter with an upper threshold 
id a lower threshold. The counter is so designed that it 
uunts up r steps when a one is received, and down s 
eps when a zero is received. When the upper threshold 
reached, the detector senses a target. When the lower 
reshold is reached, the detector senses noise. By varying 
e values of 7, s, and the positions on the upper and lower 
eshold, one obtains a large number of different sequen- 
al-observer detectors. The operation of one of these 
stectors is illustrated in Fig. 5. 
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Fig. 5 


Sequential Observer A,. At each sweep time, or unit 
me, an observation of yu, is made. If u, = 1, the sequential 
yunter, which is originally set at +0, is caused to count 
p 4; if uw, = 0, the sequential counter is caused to count 
own one. If the new value of the counter is > 16, 
4 = +0, and a target pulse is generated. If the new 
alue of the counter is < —3, then J,,,; = +0, and a 
o-target pulse is generated. The signal is measured from 
e first target pulse to the first no-target pulse following it. 
Sequential Observer B,. At each sweep time, or unit time, 
1 observation of u, is made. If uw, = 1, then l,,, = 1, + 2; 
Bn = 0, lus: = L, — 1, with the added provision that, if 
m= 7, t+: is changed to 2. If 1.:, < 0) 1,4. is changed 
. 2. The signal width is measured from the first target 
use, when 1,,,,; > 7, to the first no-target pulse following 
, when l,.; < 0. 


‘oving Average Detectors 


The class of moving average detectors makes decisions 
ised on the density of ones inside some interval. The 
idth of the interval is determined by the a przort prob- 
lity of the width of the target region. The rules for the 
-counter, or density counter for a width /, would be 
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If the sum of ones inside this “window” exceeds a pre- 
determined threshold, it is classed as a signal region, 
otherwise a noise region. 

A target beginning is sensed when one passes from 
noise to signal and a target end is sensed when one passes 
from signal to noise. The target center can then be calcu- 
lated by taking the midpoint. By varying the ‘‘window”’ 
width and the detector threshold setting, a large number 
of detectors of this class may be formed. 


CompuTER ANALYSIS 


A series of programs has been coded for Whirlwind I 
which simulate radar inputs and the several detectors 
under consideration. Using this stochastic model, measure- 
ments have been made for the purpose of evaluating these 
detectors. 

For purposes of this study, a fixed range has been 
assumed; 7.e., one of the horizontal lines in Fig. 1. The 
spread of the simulated azimuth pulses is about 9 beam- 
widths of the radar. In approximately the center of the 
azimuth spread, a target corresponding to one beamwidth 
has been simulated. 


a7} ) c d 


Fig. 6 


Fig. 6 is a representation of the simulated input. In the 
interval a = 1 to b, a Bernoulli sequence with p = P,, is 
generated; in the interval b toc — 1, a Bernoulli sequence 
with p = P, is generated; and in the interval c to d, a 
Bernoulli sequence with p = P,, is generated. For almost 
all runs, b = 80,c = 96 andd = 144. A sequence of 15-bit 
random numbers is generated, using a pseudo-random 
number generator. These are then split into two 7-bit 
random numbers. Each 7-bit random number has a value 
between 0 and 127. A test number M is chosen, and if the 
random number is less than or equal to M, a one is gen- 
erated; if the random number exceeds M, a zero is gen- 
erated. Thus, if M = 63, a Bernoulli sequence is generated 
with p = 0.5. If M = 3, a Bernoulli sequence is generated 
with p = 0.03. Thus, in this model the radar quantized 
video is simulated by the outputs of two Bernoulli se- 
quences—one with p = P,, the other with p = P,. The 
width of the target region is chosen to agree with the 
assumed beamwidth. 

This composite Bernoulli sequence is repeated a large 
number of times—for most runs, 1,000 times. The density 


34 


in the noise regions and in the target regions is given by 
the binomial distribution. 

The several detectors were simulated by programming 
Whirlwind I to carry out the actual operations of the 
detector. Briefly, for each of the 1,000 runs of simulated 
radar data, the coded detector acting on the basis of the 
incoming zeros and ones made decisions as to target or 
noise. When a target was detected, the center was located 
with reference to the units a = 1 tod of the particular run. 
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Fig. 7 


For example, the flow chart which is a symbolic repre- 
sentation of the operations to be performed by a program 
for the Success Run B observer is given in Fig. 7. From 
this flow chart, the coded program for Whirlwind I was 
written. In a similar way, the other detectors were pro- 
grammed. 


Number of 
Target Centers Detected 


Location 


Fig. 8—Results for Success Run B. 


For purposes of evaluation, several measurements were 
recorded and printed out. For each detector and each 
pair of settings P,, P,, the detected target centers were 
recorded in units a = 1 to d. Thus, for example, for 
Success Run B (P, = 0.03125, P, = 0.3125), the values 
given in Fig. 8 were printed out. The left-hand column 
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gives the position from a = 1 tod = 144, and the right 
hand column is the number of target centers detected a 
each location. A similar measurement of target-beginnin 
locations was made for some detectors. The total of th 
right-hand column gives the number of targets detecte 
in 1,000 runs. 

Using a desk calculator, the mean and variance of th 
distribution of target center locations was determined. 

It is possible, of course, that a detector may rese 
inside a target region and thus detect one target as tw 
targets. Therefore, the number of times no targets, on 
target, two or more targets were detected was recorded. 

For purposes of evaluating the radar input model, th 
density in the intervals a < « <6 —1,b = 2 S ¢ =m 
ec <x < d— 1 were measured. Also, the frequency c 
ones at any given unit was recorded. Several other auxi 
lary measurements such as these were made to check ou 
the coded programs. 

Data obtained are presented by graphs [Figs. 9 and 1 
(opposite) and Figs. 11-13 (p. 36)], one for each detecte 
and each setting of P,,, P,. The graphs give the distributio 
of target-center locations with reference to the center ¢ 
the generated signal region; 7.e., if the target region is 8 
<a < 95, the center is taken to be x = 88. The followin 
information is included with each graph. 

1) The values of P,, P,, and a (see Target Description 

2) The value of Q (the percentage of targets detectec 
the probability of target detection, or blip-scan ratio); 

3) The value of FA (the probability of false-targe 
detection); 

4) The value of ST (the probability of detecting a sing] 
target as two or more targets); 

5) The values of m and oc, (the mean and variance of th 
distribution of target-center decisions). 


THEORETICAL CALCULATIONS 


The equations for the probabilities of the various state 
of the L-counter of the success-run observer were writte 
out. A detailed study of this kind of analysis has bee 
made by Reed.’’® In order to illustrate the method, th 
equations for Success Run A will be written out in detai 
as follows: 


0 
Un+1 = Pr&tn 


2 cl 0 
Un+1 ca Pn&n SF Dra Givn=1 


Do Dal, He Dene 

Lt = 40 ap ee ++ D-8Dn-2Un= tae 
Tria =" Dalia | Dita nee Oe 

Envi = "Dally + D,_10 beet ep aed G ee 

Ent, = Delgo Dp td eee en ae 

Tew = Dale, “AOD Ga Se eee ee 


+ Pn=34n-29n—1 ete 


OTe S. Reed, Lincoln Lab. Tech. Rep.. (not generally available). 
®T. S. Reed, Lincoln Lab. Tech. Rep. (not generally available 
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where 2,,, is the probability that the counter has the 
value 7 at time n + 1, and p, is the probability of a one 
at time n. These equations are derived by assigning 
probability measures to the sets of Bernoulli sequences. 

In a heuristic manner, the equations are derived by 
considering the reset conditions and writing down the 
probability of the other events. For example, to get 
2,41, one knows that the count must have been 4 at time n. 
The reset conditions for count 4 are 0 0 0. Thus, a one in 
the first place will always cause the counter to advance 
(this corresponds to p,x,). A zero in the first place and a 
one in the second place will give x... (this corresponds to 
Q@Pu-1t,-1). A zero in the first two places preceded by a 
one will give x2,, (this corresponds to G.Qn—1Pn—2%n-2). Thus, 


ap? 
Un +1 


4 as) me 
= Pnr&n aa Pn—-1UnUn-1 si (OOO fy OO 


By similar reasoning, the other equations may be derived. 
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In order to solve the transient problem, that is, the 
determination of the probability of detection of a signal : ‘ ; Lee ate ’ oe 
region following a long noise region, it is necessary first to v= pr rpg =px +pqe =p + ge 
calculate the steady-state probabilities in noise, which x* = px + pox? + p'q?e® 
are then used as the initial conditions for the transient 
equations. a = pe + pax. + pga 
The steady-state equations are obtained from Success : ; . ae 
Run A by removing the subscripts n which refer to time, c= px + pqe + pyx 
since it is now assumed that these probabilities are inde- 2! = pe’ & pon” pat 
pendent of time. We then obtain the following steady- 
state equations for Success Run A. x = pu’ + pqx’ + py’x® + pga 
et 0 Sar 8 7 26 3.5 
a = pr c= pe + pay +pge + pax 


v= pe + par =px + pax = pr & = px + pqn’. 
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Fig. 15—False alarm rates. 


To solve these equations, we sum the columns of this 
infinite system, using the identity that }°?_, x’ = 1, and 
obtain 


1-2 =pt+pqtop¢e — 2 — 2’) 
te fae pee eee erie I ley eats 
or 


4 
0 


ee oe 

With the value of x) obtained by substituting numerical 
values of p = P,, and gq, the values of x’ for steady-state 
noise can be obtained. Then the procedure is to calculate 
Oe = 11, 2+ 8 if S, = >);-7, 21, or the probability 
that the counter has a value exceeding 6. Assuming that 
within a signal region no reset for 1 > 6 occurs, S,., = 
S, + x°.,. Therefore, 8S, = S) + xj. Finally, 2] = 1 — 
>-%_, «i — S,. The second column is computed in just 
the same way. The value of S,, is the probability that the 
count has value exceeding 6 at time n = 16 or the prob- 
vbility of detecting the signal. These calculations were 
carried out using a desk calculator. The results are given 
n Fig. 14. 
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BLIP-SCAN RATIO 
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Fig. 16—Target detection curve. 


TARGET-CENTER DECISION BIAS 


Fig. 17—Target center decisions. 


Information about the false-alarm rate can be obtained 
from the steady-state equations for noise. This was done 
for various values of P,, and the results collected in Fig. 15, 
where 7% is the probability of detecting a false target. 

The equations for any other detector in this class may 
be written in just the same way and similar calculations 
made. It is possible to program these equations on Whirl- 
wind I. 


CoNCLUSION 


A good detector should have a high blip-scan ratio 
for targets and a low false-alarm rate; 7.e., sharp discrimina- 
tion between target and noise. This will be referred to as 
the detection criterion, and may be represented graphically 
by a curve of blip-scan ratio vs P, for a fixed P,, or a, as 
in Fig. 16. For good discrimination, the detection curve 
should have a sharp knee. 

A good beam-splitter should have a decision bias which 
is very nearly independent of signal strength and has a 
very narrow variance. This will be referred to as the 
beam-splitting criterion, and may be represented graphi- 
cally by a curve of decision bias vs P, for fixed P,, or a, 
as in Fig. 17. 

A prohibitive amount of calculations would be required 
to obtain curves for both detection criteria and beam- 
splitting criteria for each detector. Therefore, it was 
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necessary to extrapolate from just a few points of these that it does not seem expedient for most applications to 
curves. In most cases, results were obtained for two complicate the detector for the very small increment in 


values of P, (0.3 and 0.5) for P, = 0.03. These results accuracy. Of course the resolution time of these detectors 
have been shown graphically. is limited to a beamwidth. 
The analysis described here consisted primarily of a com- Although a Monte Carlo method using a high speed 


puter study using a Monte Carlo Method. Sufficient exact digital computer provides information about proposed 
analysis was carried out to compare certain check points. detectors, the solution of the statistical problem would be 
By these methods, any proposed beam-splitter could be more useful in predicting the performance of new detectors 
analyzed. A very large number of detectors were studied and in evaluating the influence of parameters such as” 
which are not reported here. Furthermore, consideration beam-width, signal strength, or beam pattern. | 
was given to other methods employing amplitude informa- 
tion or beam shape. The general conclusion is that the 
moving window detector satisfies the detection and beam- The helpful advice and criticism received from many 
splitting criteria and at the same time is logically the associates at the Lincoln Laboratory is gratefully ac- 
simplest. A moving window detector can provide accuracy knowledged. A special acknowledgment is due Miss B. | 
of target center with a variance of two or three units, Jensen for carrying out the extensive numerical calculations| 


or sweep times. This is so close to the theoretical optimum and for the preparation of many of the figures. 3 
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Inverse Probability in Angular-Tracking Radars’ 


B. J; DUWALDT; , 


Summary—An angular-tracking radar of the pulsed type is ana- distribution’ * and, further, that the amplitudes are, 


lyzed for targets whose signal-return in the video follows a Rayleigh . : ; 
distribution. The pulse amplitudes are assumed independent from independent from pulse to pulse. The noise out of the 


pulse-to-pulse but closely correlated for the duration of a pulse detector between pulses will, of course, also have a Rayleigh 
width. White Gaussian noise is introduced into the input of the distribution.’ When operating in the tracking mode, the 


receiver. It is assumed that rf phase information is lost. The method _ d ill oy. aera 

of inverse probability is used to find the most probable direction of radar antenna Wl can in the vicinity of the target such, 

the target from the scan axis. It is found that interference may be that the mean-square signal-to-noise ratio at the detector 

reduced by synchronizing transmissions in the scan cycle, by trans- oytput will be varied in accordance with the angular 

mitting pulses in pairs simultaneously or nearly simultaneously, a ; . : 

sitio y acrensine whetnumben or aleecin tee ted! position of the target with respect to the scan axis. With 
a knowledge of the beam-shape and pulse-transmission | 


INTRODUCTION locus, the probability distribution of target angular 
dies RADAR considered here is a pulsed radar position may be derived from observation of the detector 


system in which the rf phase information is lost. output, henceforth designated y. This a posteriori prob-. 
The receiver is assumed to consist of linear, band- bility distribution is found from the relationship 


limiting mixer-IF stages and an ideal linear envelope 
detector. White Gaussian noise is introduced in the first 


stage of the receiver. . ts L. ee and G. Ei. Uhlenbeck, “Threshold Signals,’ M.1.T 
: ‘ aa ec 
It will be assumed that the nature of the targets is such York: 1950. aboratory Series, vol. 24, p. 249, McGraw-Hill, New 


that the received pulse amplitudes will have a Rayleigh ? D. KE. Kerr, Ed., “Propagation of Short Radio Waves,” M.I.T. 
Rad. Lab. Ser., vol. 13, McGraw-Hill, New York, Chapt. 6; 1951. 
3 R. H. Delano, “A Theory of Target Glint or Angular Scintillation 
* This paper is based on part of a doctoralthesissubmitted August, in Radar Tracking,’’ Convention Recorp or THE IRE, part I, p. 13; 
1955 to the faculty of Purdue University. The work was supported 1953. 


by the U. 8S. Naval Ordnance Plant, Indianapolis, Indiana, under 4 P. Swerling, “Probability of Detection for Fluctuating Targets,” 
Department of the Navy Contract No. Nonr 1100-(03). RAND Corp. Res. Memo. RM-1217; March 17, 1954. 
+ Ramo-Wooldridge Corporation, Los Angeles, California. This °S. O. Rice, “Mathematical Analysis of Random Noise,” Selected 


work was done while the author was a Research Instructor in the Papers on Noise and Stochastic Processes, p. 214, Dover Publications, 
School of Electrical Engineering, Purdue University, Lafayette, Inc., New York; 1954. This paper originally appeared in Bell Sys. 
Indiana. Tech. Jour., vols. 23 and 24, 1945. 
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A Posteriori Probability Distribution of Position 
= A Priori Probability Distribution of Position 
X Likelihood Function of y 


A Priori Probability Distribution of y 


(1) 


where the likelihood function of y is the probability dis- 
tribution of y given a target position.* ’ 

To use the method of inverse probability, it is first 
necessary to consider the a priori distribution of the 
pulse amplitudes. The beam power pattern is assumed 
to yield a mean-square signal-to-noise ratio of 


= 40" 
(Oh => Ohana exp ya ) 


(2) 


where nx 1s the mean-square signal-to-noise ratio when 
the target is on the beam axis, 6 is the angular distance of 
the target from the beam axis, and B is a measure of 
beamwidth. This signal-to-noise ratio assumes that pulse 
transmission and reception occur at the same angle with 
respect to the beam axis. The signal-to-noise ratio is 
¢/o° where 2¢° is the mean-square value of y when there 
is a target but no noise and 20° is the mean-square value 
of y when noise alone is present. 

The antenna beam axis during transmission and recep- 
tion always lies on the surface of a cone whose axis is the 
scan axis. The angle between the scan axis and beam 
axis 1S 0). The target lies on a sphere having its center at 
the radar antenna. The surface area on this sphere cir- 
cumscribed by the locus of the transmission beam axis 
may be approximated, for small 4, by a plane surface 
perpendicular to the scan axis and through the target. 
The angles between the beam axis, scan axis, and target 
line-of-sight will then be approximately directly pro- 
portional to the line segments joining the intersections 
of these axes on the plane and the target as long as the 
angles are small (say less than 20°). The plane is shown 
in Fig. 1 in which F is the range of the target from the 
radar antenna. 

From the geometry of the figure, 

6 = 6 + A’ — 26,A cos (¥, — 4), (3) 
where A is the angular distance of the scan axis from the 
target, y, is the angular rotation of the beam axis from 
the reference line at time ¢, (the time of transmission), 


and ¢ is the angle of the target from the reference line. 
From (3), 


aA, >, Yn) 
eA i 2 
= Omax exp yee [0 —- A cs 20,A COs (Vn a ¢)]. (4) 


The signal-to-noise ratio a) when the target is on the 
Scan axis 1S 


6 P. M. Woodward, and I. L. Davies, “Information theory and 
inverse probability in telecommunications,”’ Proc. I.E.E., vol. 99, 
part III, p. 37; March, 1952. 

7P. M. Woodward, “Probability and Information Theory, with 
Applications to Radar,” McGraw-Hill, New York; 1953. 


Du Waldt: Inverse Probability in Angular-Tracking Radars 


39 


Qo = Amax EXP = 1.465 
Oa max 2 Dre 
1B? 


(5) 
Hence, (4) may be written 


—1.4A? 


2.80,A 
iB = , 


B08 a — 6 | (6) 


a(A, , Wn) = Ao exp) 


Beam Axis 
ee at time ft, 


Reference 


Beam Locus 


Fig. 1—Plane approximation of target area on the tracking sphere. 


A Priort PRoBABILITY DISTRIBUTION 
or Putse AMPLITUDE 


One difficulty that arises in analyzing a tracking system 
is that the antenna is part of a closed loop. The amount 
the scan axis is likely to be off the target depends upon 
how well deviations of the scan axis from the line-of-sight 
are detected and how well the control system returns the 
scan axis to the line-of-sight. Hence, the a priorz prob- 
ability distribution of pulse amplitude will be a function 
of the error constants of the servo system in the tracking 
loop. For a first-order analysis, however, it may be 
assumed that the radar does a good job in keeping A very 
small. It is reasonable to assume, then, that the a priori 
probability distribution of pulse amplitude will be a 
Rayleigh distribution resulting from a mean-square 
signal-to-noise ratio dy. If k pulses are considered, let 
fy} = (Yi, Yo) *** 5 Yx) represent the received signal from a 
set of k pulses. Since there is no dependence between 
pulses, the joint a@ priori probability of having this set of 
k pulses is simply the product of the probability distri- 
butions of the individual pulses. Hence, for Rayleigh- 
distributed pulses, the a priori distribution becomes 


n=k 


I] we —Yn | 
fa + aol b al a ee 


n=k 


pliynt] = 


8The a priori probability distribution of pulse amplitude for 
some particular parameters is presented in the Appendix of the 
thesis cited. 
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LIKELIHOOD FUNCTION AND A POSTERIORI PROBABILITY 
DISTRIBUTION OF TARGET ANGULAR POSITION 


In a like manner, the likelihood function of { 
A and ¢, is 


Yn} given 


Pa.oltYn} ] aa 


n=k 

| Yn 

n=1 
n=k 


T] o°l1 + ala, ¢, ¥)] 


aS =t | 
sit bP + a4,é wil © 

The a@ priori joint probability distribution of (A, ¢) 
depends upon the nature of the targets and the radar 
system tracking loop. It seems likely that with land-based 
radar tracking aircraft targets there will be more error in 
bearing than in position angle. On the other hand, a radar 
system on a rolling ship may have more error in position 
angle than in bearing. Hence, the most likely assumption, 
in general, is a flat distribution for ¢@ and a Rayleigh 
distribution for A 

The a posteriori joint probability distribution of (A, ¢) 
is, substituting (7) and (8) into (1), 


Ee ON eee 
I [1 + a(A, 4, ¥,)] 


Piun\(A 


exp | 5 ynla(A, 6) vn) = do] | 


LEIP KET CE SRI ee 


If the a priori distribution of A is such that A is almost 
always very much smaller than B, then a(A, ¢, y,) may 
be approximated by 


aA, >, Wn) = ae a: cos (Yn — |. (10) 


If a further simplification is made by using a, for 
a(A, ¢, ,,) except where the difference of these two quan- 
tities occurs, then the a posteriori distribution becomes 


n=k 


Piel (ape ctcl| Se ee 


ae cos (¥, — |. (11) 


DECISION ON DIRECTION OF THE TARGET 
FROM THE SCAN AXIS 


If there is no reason to weight the decision on ¢, the 
most probable value may be used. The value of ¢ which 
maximizes the exponential is found by differentiation. 
The exponential is maximum when its argument is maxi- 
mum. Ignoring the multiplier which is independent of 
n and ¢, the function to be maximized is 
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n=k 


H = Do y, cos (Yn, — 9) 
n=k n=k 
= yz cos y, cosd + > yzsin y, sin ¢ (12) 
= X cos¢ + Ysin¢, 
where 
n=k . 
x = Yn COS Pp, 
Sige (13) 
n=k 
Y= D>) y, sin 


Differentiating with respect to ¢, and equating to zero, 


a Y cos? — X sing = 
gives a maximum when 
cos d = ess 5 | 
VX? + Y (14) 
‘ Y. 
SE es | 


where the square root is positive. The ¢ specified by (14) 


is, therefore, the most probable value. | 


QUALITY OF THE DECISION | 

Assuming that (14) is the best decision on ¢, the quality 
of the decision depends upon parameters such as pulsing 
pattern, beamwidth, scan angle, and number of pulses. 
The first part of the criterion of quality to be used here will 
be that the average values of X and Y simultaneously 
become directly proportional to cos ¢o and sin ¢o, respec- 
tively, where ¢, is the true value of ¢ of a target whose 
position is being estimated. The second part of the criterion 
is that the standard deviations of X and Y be made small 
in comparison to the means, as the means assume their 
desired values. 

In examining the quality of the decision, only X will 
be analyzed, but the analysis of Y is essentially identical. 
The choice of radar parameters to optimize X will also 
optimize Y. 


MINIMIZATION OF INTERFERENCE 


Consider a target at (Ao, ¢) with respect to the scan 
axis which returns a signal-to-noise ratio a(Ao, do, ¥,,). Let 


2 
Yn 
LS . 15 
2a*[I + alAs, do) VI i 
Then 2, = 3k. where x, has a chi-square distribution of 


two degrees of freedom. The mean of z, is then one, as ls 
also the variance. Using the approximation of (10), 
X may be written 


1956 . Du Waldt: Inverse Probability in Angular-Tracking Radars 41 
n=k . 
2.86,A 
¥ = a 2e"e| 1 Sigpilaet Qo Bo oct eeaa 6) | Hein pss third term may be ats 
n= n= n=k/2 
its > cos* yb” cos’ ¢ = >> (cos* ¥, + sin’ ¥,)b? cos? by 
n=1 n=1 
= 2o°(1 + a) yy. 2,(cos WY, + cos y, sin y,b sin do a 
n=1 x 
= k/2 db’ cos’ d) — 4 >> cos’ y, sin? yb” cos’ do, (22) 
+ cos’ ¥,b cos do), (16) sg 
h where the n’s are chosen from the appropriate quadrants. 
yeecre Combining terms, 
b na Ao 2.80 Ao (17) oe = [20°(1 — ao) | [k/2(1 + b” cos’ do) 
1 + Qo 18s" n=k/4 
; ; et 4b? O27, ms ae 2 Ob ah 9 
Since the z,’s are independent, the mean value of X is isin! x eae 2 


& n=k 
X = 20°(1 + a) >> (cos ¥, + cos y, sin ¥,b sin do 


+ cos’ y,b cos ¢o). (18) 


In accordance with the chosen criterion of quality, 
the first two terms following the summation sign are 
interference and should be minimized. The first term is 
made zero if the pulses are oppositely paired. That is, 
for every pulse transmitted at y,, one is also transmitted 
at y, + mw. The second term, which may be considered 
cross-talk, is made zero if pulses are arranged in quadruple 
sets. Then, for each pulse at y,, pulses are also trans- 
mitted at y, + 7/2, ¥, + 7, and y, + 32/2. With this 
pulse pattern, X becomes 


n=k/2 


20°(1 + a) Dd) (cos’ y, + sin” ¥,)b cos do 


n=1 


) 


= 20°(1 + a) k/2 b cos do. (19) 
Now consider the variance of X. 
n=k 
ox = [20°(1 + a)]’ >> (cos ¥, + cos ¥, sin ¥,b sin 


+ cos’ y,b cos $o)” 


[2o°(1 + ao)] > (cos’ y, + cos’ y, sin” y,b’ sin’ do 


+ cos* y,b? cos’ d. + 2 cos® Wb cos do 
+ 2 cos’ y, sin y,b sin ¢o 
+ 2 cos® y, sin y,,b’ sin ¢o COS ¢o)- (20) 


For the quadruple-set pattern of pulsing, the last three 
terms of ox become zero. The first term becomes k/2. The 
second may be written 


n=k 
> cos’ y, sin’ y,,b sin’ ¢o 
n=1 


n=k/4 


= 4.>° cos’ y, sin’ y,b’ sin’ ¢o, 


n=1 


(21) 


where, for n from 1 to k/4, the y,’s are in one quadrant. 


The last term in (23) can be positive, negative, or zero, 
depending upon ¢ ). However, the corresponding term 
which arises in the analysis of Y has the opposite sign. 
Therefore, the term ought to be eliminated. It will be, if 
pulses are transmitted only at 0, 7/2, 7, and 37/2. These 
four values of y, are optimum for transmission, if such 
an arrangement does not limit another parameter. In a 
system where the time consumed in switching the beam 
axis from one value of y, to another is not negligible, the 
number of pulses & available for an observation are 
limited. In such a system, the necessity for increasing k 
makes it necessary to choose adjacent y,’s close together. 
In any event, because of our prior assumption of very 
small A/B, b would be small enough to make the latter 
term in the brackets very small in comparison to the 
first term. 

Assuming, however, that pulses are transmitted only 
at 0, 1/2, x, and 37/2, the ratio of standard deviation 
to the mean becomes 


Ore ND Ne COS” bo 


Oe Wk b cos oo 


This quantity is reduced with increasing k and increasing b. 
The assumption that was made in approximating 
a(A, $0, Wn) also makes b very small in comparison to one. 
Hence, how large we can make b and still improve per- 
formance depends upon an analysis with a better approxi- 
mation of a(A, ¢, ¥,). The method of decision in (14) 
could be tested for validity in the case of large b by using 
the exact expression of a(A, ¢, ¥,) in (16) and the subse- 
quent analysis. A useful series for a(Ao, ¢0, ¥,) in that 
case would be 


(24) 


1.4A5 
a(Ao, Po, va) = ofp exp aa JEP 


| 10 12% x I(r) cos p(vn — 6) | (25) 


where J,(r) is the modified Bessel function of the first 
kind and 
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RB (26) 


T= 


However, such a test will not be made here. 
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At any rate, if b is very small, there is a tendency 
toward improvement if b is increased. This can be done 
for constant A, by increasing 6, and decreasing B, as 
long as ad) (which is a function of 6) and B) is kept large 
in comparison to one. 


REDUCTION OF INTERFERENCE BY PULSE CORRELATION 


An additional method of reducing interference is sug- 
gested, in the case of a0 — 2/2 — a — 32/2 pulse pattern, 
by writing X in this fashion: 


t=k/4 


X = 207°(1 + a) >> 2b cos - 1) 


i= 


7=k/4 


+ > z(bcos¢o-— 1, (27) 

j=l 
where 7 is for times when cos y, = 1 and 7 is for times 
when cos y, = —1. If pulse pairs could be completely 


correlated so that 2; = z2;, the mean of X would become 


XxX = 20°(1 + Ao) k/2 b COS do- (28) 
This is the same as (19). The variance is now 
ox — [20°(1 + a) |” k b? cos” do, (29) 
and the ratio of the standard deviation to the mean 
Ox y 
SS == 30 
X | Vk oe 
Correspondence 


The Linear, Input-Controlled, 
Variable-Pass Network 


The writer, finding himself unable to 
establish the validity of Theorem I-A of 
this paper! along the lines of the proof 
outlined there (integration by parts and 
use of the mean value theorem for integrals), 
wishes to submit an elementary proof of this 
theorem under more general conditions. 


stated as follows. 


fins De’ 


Original Statement of Theorem I-A 


If F(t) is a bounded, continuous, single- 


The result of this communication may be 


If F(t) is a bounded, continuous, single- 
valued function of ¢, then 


1 T 
ine / F(t) dt 
af 


yi t 
= lim ¢ [ iF 
Too sh —T —-o 
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For small 6, this is an improvement over (24). 

It is possible that close correlation could be established 
by simultaneous, or nearly simultaneous, transmission of 
pulses at opposite sides of the beam locus. Even if the 
received signals were completely correlated in amplitude 
for opposing-pulse pairs, however, the added noise would 
still be uncorrelated. Therefore, ¢,/X would be larger than 
(30), depending upon the magnitude of a and b cos ¢o, 
but smaller than (24). 


CONCLUSION 


The conclusions to be drawn from the above discussion 
are: 1) Interference due to noise and signal fluctuation is 
reduced if pulses are paired to be transmitted on opposite 
sides of the beam-axis transmission locus. 2) Interference 
due to cross-talk is reduced if pulses are transmitted in 
quadruple sets and will be zero if pulses are transmitted 
only at y, = 0, 7/2, 7, and 37/2. 3) Interference inde- 
pendent of the position of the target is reduced if the 
pulse-pairs are transmitted simultaneously or nearly 
simultaneously. This might be accomplished with a rapid 
lobe-switching scheme. 4) The accuracy of position 
determination increases with the number of pulses 
integrated. Because of relative motion between the scan 
axis and the target, several radars might have to be used 
together to increase the effective number of pulses. 


I(T) = 1(T) + 1(1), (2) 
New Statement of Theorem I-A eee | 
I(T) = - i! F(xje **"” da dt “(Gq 
Di | 
and 
hes 7 | F(x)e?*" da dt. (4 
F(ae °°"? da dt bs ) 


Consider first the function J,(7). An 


valued function of ¢, and if its derivative, 0) Zap x eas a real. examination of Fig. 1 shows that we may 
F’(t), is continuous, then ’ : write 
Proof: Consider function (7) defined by 
; 1 E Pa oy pt pt 
lim 5a | F(t) dt . ee ee 1() = [ / ee ae’ 
ane ag RE Ps (i) TCE atest) F(a)e ?*“'~” da dt. At) Al pe [es P(a)e dx dt. 
T at ig (5) 
4 a —2a(t—z) (1) 
= lim + [ (l F(a)e dx dt I anes : : : 
EET ih pak ee am domain of integration D of I c 1) is ea eee the order of integration this 
show e 
WKS G2 SES a real. Eee 


1B. E. Keiser, ‘The linear input-controlled, vari- 
able-pass network,’”’ Trans. IRE, vol. IT-1, pp. 34- 
39; March, 1955. 


D=D,+ Dz is a convenient partitioning 
of D into subdomains as shown. By con- 
sideration of (1) and the partitioning of D 
we may at once write 


I(T) = ef iP F(x)e2*"- dt dae 
(6) 
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It is now possible to perform the integration 
with respect to ¢ to obtain the result 


BCP) = oe af. F(a)e** 


R (Gan 


a Gadde) da, (7) 
whence 

il i 
Oe i F(a) de 


| Cae 


| oT 


ie F(x)e* de. (8) 


Fig. 1—Domains of integration. 


Since by hypothesis F(x) is bounded, with 
upper and lower bounds M and m respec- 
tively, say, and a is real and positive, the 
last term of (8), which will be denoted by 
J(T'), becomes 


Mig 24 iT 
ae = il let) 9 
eo i 


CMe fi 
S 2ax , 
ae ie e"* dx, (10) 


giving the result 


JL) 


and 


J(T) 


< AGees (lmewen sya (11) 


ee 
It follows directly from (11) that 


lim J(T) = (12) 


Consider next the function [.(7), which 
from the definition of D2 may be written as 


T. —T 

n= | i F@jeue edz cde, 
ff —T — 00 

(13) 


This may at once be expressed as the 
product of two simple integrals to give 


AE 
1,(T) = leone dt 
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(15) 


Employing an argument similar to that used 
when considering J(7'), we find at once that 


oe BSH 
4aT 
M 
s1(T) < — oA? 
S1(1) <7 —e“*"). (16) 


It again follows directly that 
him (1) = 


T—02 


(17) 
From (1), (2), (8), (12) and (17) it follows 
finally that 


7 
wie lim 5a | F(z) dx, (18) 
ene eg tht 


which is the required result. 

It may be noted here in passing that this 
new statement of Keiser’s Theorem I-A is 
still far from the most general formulation. 
It has been quoted here requiring continuity 
of F(t), since the above simple proof then 
follows directly, but provided F(t) is 
Lebesgue integrable an appeal to Fubini’s 
Theorem establishes the result under far 
more general conditions on F(t). 


ALAN JEFFREY 
G.E.C. Stanmore Labs. 
Middlesex, England 


Rebuttal 


The proof offered ‘by Mr. Jeffrey is 
correct in every respect, and I raise no 
question about the validity of his comments. 
However, the’ functions which the above- 
referenced paper deals with have derivatives 
which are bounded and continuous every- 
where. Therefore, no attempt was made to 
obtain a more general proof. 

The proof of the theorem was omitted 
because of lack of space. If either the First 
or the Second Mean Value Theorem is 
utilized to obtain the proof, then, of course, 
F’(t) must be bounded everywhere, as well 
as continuous. 

Mr. Jeffrey has submitted to me a proof 
using the First Mean Value Theorem. My 
proof, which uses the Second Mean Value 
Theorem, also requires that F’(t) be 
bounded everywhere and continuous, al- 
though this restriction can be lifted with a 
proof of the type presented by Mr. Jeffrey. 

The theorem and its proof are as follows: 


Theorem I-A 

If F(t) is a bounded, continuous, single- 
valued function of ¢, and if F’(¢) is con- 
tinuous and bounded, then 
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1 iu 
lim om jis F(t) dt 
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T t 
= lim al / F(x)e?*" dx df. 
Too —-T J-o 


Proof: Consider the quantity 


t 
2a | ee ** F(x) da. 


Let F(x) = 


Since 
/[ udv = 


ry 


u and let 2ae? dr = dv. 


vy) = i! udu, 


eel Qae*** F(x) dx 
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= ctl Roe 


=o 


— i : e** F(x) as | 


eel rege 


_ le e** F'(x) ae 


Thus 


t 
coal Qae"** F(x) dx 


= F(t) — if ge ORG) dca) 


Integrate each side with respect to ¢ from 
= 0 tot = T, divide by T, and take the 
limit of each side as T — @. This gives 


1 t 
lims= [ oe / 2ae"** F(a) dx dt 
0 —o 


4: pica 
Fad ed | (2) 


—lim = or 
fol 
The following is a proof that, under the 

conditions postulated, 


ee a) dx dt. 


lim an [ie Pati? F(a) dx di=0 
TO0 
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First note that 


lim oie [ie mebananbaaa KA C2) Foi: 
Fae! li 

==) lim af a 

eel 


ere e°* F(x) ac| dt. (4) 
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Consider the term in brackets on the right- 
hand side, 


t 
[ e°7 F(x) dz. 


There is a mean-value theorem for integrals? 
which states: 

Let f(x) and g(x) be two functions which 
are continuous in (a, b). If ¢(z) is a positive, 
monotonic function, increasing in (a, 6), then 
there exists a value £, wherea < — < b, such 
that 


/ f(x)e(a) dx = ¢(b) I f(x) dx. (5) 


Let g(x) = e? and let f(z) = F’(x). Take 
aas —o and b as ¢t. Then the theorem 
quoted above states that there is a & 


between — © and ¢ such that 


; [ Pa) dx. 
(6) 


Integration of the right-hand side yields 


t 
[ e F(a) dx =e 


t 


Je e°* F'(z) he e*'[ F(t) — F(é)]. 


(7) 


Substituting this relationship into the right- 
hand side of (4) gives 


S. Sokolnikoff, ‘Advanced Calculus,” 
Hill Took Co., Ine., New York, N. Y., 


McGraw- 
p. 115; 1949. 
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If F(t) is a constant for all values of t, then 
F(é) must be this same constant, since 
F(~) merely takes on values which F(t) 
took on previously. In this case, the right- 
hand side of (8) is zero. Suppose F(t) is not 
a constant, but is the sum of a constant 
term and a term varying with ¢. Then 
F(&) is of the same nature. In particular, 


F(t) = A+ B(t) (9) 
Fé) = A+ BO, (10) 


where the average value of B(é) over all ¢, 
or of B(é) over all € or ¢ is zero, from the 
way in which the constant A was chosen. 
Then 


=e) lea 


Tr 
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lima f FO- FOL gy 
iT 
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From (2), it follows that 
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Since F(t) is assumed to be a bounded, | 
continuous function of ¢ for all values of ¢, | 
the integral on the left-hand side of (13) | 
also exists if its limits are taken as —T to T. | 
t_, e-2(t-*) F(x) dz is finite’ 


Furthermore, 
for all values of ¢. Therefore, in the integra- 


tion with respect to ¢ on the right-hand side | 


of (13), the limits 0 to 7 may be replaced 
by —T to 7, and the resulting integral still 
will converge. Thus, in view of the fact 
that F(é) is bounded for negative values of 
t as well as for positive values of ¢, it is 
permissible to write 


lim 5 af F(t) dt 
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ACTIONS. Papers submitted should include a statement as to whether the material 
has been copyrighted, previously published, or accepted for publication elsewhere. 


Papers should be written concisely, keeping to a minimum all introductory 
and historical material. It is seldom necessary to reproduce in their entirety previ- 
ously published derivations, where a statement of results, with adequate references, 
will suffice. 


To expedite reviewing procedures, it is requested that authors submit the 
original and two legible copies of all written and illustrative material. The manu- 
script should be double-spaced, and the illustrations drawn in india ink on drawing 
paper or drafting cloth. Each paper should include a carefully written abstract of 
not more than 200 words. Upon acceptance, papers should be prepared for publica- 
tion in a manner similar to those intended for the PrRocEEDINGS oF THE IRE. 
Further instructions may be obtained from the Publications Chairman. Material 
not accepted for publication will be returned. 
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N. J. Local Chapter activities and announcements, as well as other nontechnical 
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